欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

LeNet-5论文笔记

程序员文章站 2024-03-14 11:06:34
...

LeNet-5:Gradient-Based Learning Applied to Document Recognition

1998年的LeNet-5是CNN的经典之作,但是该模型在后来未能火起来,主要原因是当时的计算力不足和数据量的不足。并且当时的SVM在可承受计算量的情况下,达到,甚至超过了神经模型。CNN虽然在当时没能受到重视,但是并没有掩盖CNN的强大能力。

下图是LeNet的架构图:

LeNet-5论文笔记

LeNet由两层conv,两层pool,三层fc组成。

以下是用TensorFlow和Keras混合编写的LeNet-5模型:


import tensorflow as tf
keras = tf.keras
from tensorflow.python.keras.layers import Conv2D,MaxPool2D,Dropout,Flatten,Dense

def inference(inputs,
              num_classes=10,
              is_training=True,
              dropout_keep_prob=0.5):
  '''

  inputs: a tensor of images
  num_classes: the num of category.
  is_training: set ture when it used for training
  dropout_keep_prob: the rate of dropout during training
  '''

  x = inputs
  # conv1
  x = Conv2D(6, [5,5], 1, activation='relu', name='conv1')(x) 
  # pool1
  x = MaxPool2D([2,2], 2, name='pool1')(x)
  # conv2
  x = Conv2D(16, [5,5], 1, activation='relu', name='conv2')(x)
  # pool2
  x = MaxPool2D([2,2], 2, name='pool2')(x)
  x = Flatten(name='pool2_flatten')(x)
  if is_training:
    x = Dropout(rate=dropout_keep_prob)(x)
  # fc3
  x = Dense(120, activation='relu', name='fc3')(x)
  if is_training:
    x = Dropout(rate=dropout_keep_prob)(x)
  # fc4
  x = Dense(84, activation='relu', name='fc4')(x)
  # logits
  logits = Dense(num_classes, activation='softmax')(x)
  return logits

if __name__ == '__main__':
  x = tf.placeholder(tf.float32, [None, 784])
  images = tf.reshape(x,[-1,28,28,1])
  labels = tf.placeholder(tf.float32, [None, 10])
  dropout_keep_prob = tf.placeholder(tf.float32)
  logits = inference(inputs=images,
                    num_classes=10,
                    is_training=True,
                    dropout_keep_prob=dropout_keep_prob)

  with tf.variable_scope('costs'):
    cost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(
        logits=logits, labels=labels), name='xent')

  with tf.variable_scope('train'):
    train_op = tf.train.AdamOptimizer().minimize(cost)

  from tensorflow.examples.tutorials.mnist import input_data
  mnist_data = input_data.read_data_sets('./mnist_data', one_hot=True)

  acc_test = tf.divide(
      tf.reduce_sum(keras.metrics.categorical_accuracy(labels,logits)),
      len(mnist_data.test.labels))

  with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    Writer = tf.summary.FileWriter('./tmp',sess.graph)

    for i in range(200):
      batch_x,batch_y = mnist_data.train.next_batch(50)
      _, c = sess.run([train_op, cost], feed_dict={
          x:batch_x, labels:batch_y, dropout_keep_prob:0.5})
      acc_test1 = sess.run(acc_test, feed_dict={
          x:mnist_data.test.images, labels:mnist_data.test.labels, dropout_keep_prob:1})
      print('step:%04d  cost:%.4f  test_acc:%.3f'%(i+1,c,acc_test1))

代码以Keras来实现模型的inference过程,其他部分使用Tensorflow,这样可以大大减少构建模型的复杂度。
LeNet-5的具体配置:

配置
conv1 5x5, 6 stride 1
pool1 2x2 maxpool, stride 2
conv2 5x5, 16 stride 1
pool2 2x2 maxpool, stride 2
flatten
dropout rate 0.5
fc3 120
dropout rate 0.5
fc4 84
softmax 10

注意:使用本博客的代码,请添加引用