LeNet-5论文笔记
程序员文章站
2024-03-14 11:06:34
...
LeNet-5:Gradient-Based Learning Applied to Document Recognition
1998年的LeNet-5是CNN的经典之作,但是该模型在后来未能火起来,主要原因是当时的计算力不足和数据量的不足。并且当时的SVM在可承受计算量的情况下,达到,甚至超过了神经模型。CNN虽然在当时没能受到重视,但是并没有掩盖CNN的强大能力。
下图是LeNet的架构图:
LeNet由两层conv,两层pool,三层fc组成。
以下是用TensorFlow和Keras混合编写的LeNet-5模型:
import tensorflow as tf
keras = tf.keras
from tensorflow.python.keras.layers import Conv2D,MaxPool2D,Dropout,Flatten,Dense
def inference(inputs,
num_classes=10,
is_training=True,
dropout_keep_prob=0.5):
'''
inputs: a tensor of images
num_classes: the num of category.
is_training: set ture when it used for training
dropout_keep_prob: the rate of dropout during training
'''
x = inputs
# conv1
x = Conv2D(6, [5,5], 1, activation='relu', name='conv1')(x)
# pool1
x = MaxPool2D([2,2], 2, name='pool1')(x)
# conv2
x = Conv2D(16, [5,5], 1, activation='relu', name='conv2')(x)
# pool2
x = MaxPool2D([2,2], 2, name='pool2')(x)
x = Flatten(name='pool2_flatten')(x)
if is_training:
x = Dropout(rate=dropout_keep_prob)(x)
# fc3
x = Dense(120, activation='relu', name='fc3')(x)
if is_training:
x = Dropout(rate=dropout_keep_prob)(x)
# fc4
x = Dense(84, activation='relu', name='fc4')(x)
# logits
logits = Dense(num_classes, activation='softmax')(x)
return logits
if __name__ == '__main__':
x = tf.placeholder(tf.float32, [None, 784])
images = tf.reshape(x,[-1,28,28,1])
labels = tf.placeholder(tf.float32, [None, 10])
dropout_keep_prob = tf.placeholder(tf.float32)
logits = inference(inputs=images,
num_classes=10,
is_training=True,
dropout_keep_prob=dropout_keep_prob)
with tf.variable_scope('costs'):
cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(
logits=logits, labels=labels), name='xent')
with tf.variable_scope('train'):
train_op = tf.train.AdamOptimizer().minimize(cost)
from tensorflow.examples.tutorials.mnist import input_data
mnist_data = input_data.read_data_sets('./mnist_data', one_hot=True)
acc_test = tf.divide(
tf.reduce_sum(keras.metrics.categorical_accuracy(labels,logits)),
len(mnist_data.test.labels))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
Writer = tf.summary.FileWriter('./tmp',sess.graph)
for i in range(200):
batch_x,batch_y = mnist_data.train.next_batch(50)
_, c = sess.run([train_op, cost], feed_dict={
x:batch_x, labels:batch_y, dropout_keep_prob:0.5})
acc_test1 = sess.run(acc_test, feed_dict={
x:mnist_data.test.images, labels:mnist_data.test.labels, dropout_keep_prob:1})
print('step:%04d cost:%.4f test_acc:%.3f'%(i+1,c,acc_test1))
代码以Keras来实现模型的inference过程,其他部分使用Tensorflow,这样可以大大减少构建模型的复杂度。
LeNet-5的具体配置:
层 | 配置 |
---|---|
conv1 | 5x5, 6 stride 1 |
pool1 | 2x2 maxpool, stride 2 |
conv2 | 5x5, 16 stride 1 |
pool2 | 2x2 maxpool, stride 2 |
flatten | |
dropout rate | 0.5 |
fc3 | 120 |
dropout rate | 0.5 |
fc4 | 84 |
softmax | 10 |
注意:使用本博客的代码,请添加引用
上一篇: C++11的新特性