欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页

tensorflow 优化 MNIST数据集分类 Jupyter

程序员文章站 2022-07-13 10:30:36
...

  上篇文章简单实现了mnist数据集的分类,并在文章最后提到了对神经网络的优化,我们可以对基本神经网络进行优化,得到更加准确的数据分类。

  • 先来看一看可以进行优化的地方,我们将在不使用卷积神经网络的前提下对代码进行优化:
  1. 可以测试将mnist数据集按不同批次分类,代码中是20,可以改为30、50、100等,然后测试发现准确率最高的批次
  2. 可以优化代价函数,将基本的二次代价函数变为交叉熵释然代价函数:loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))
  3. 使用梯度下降以外的其他优化器,例如train_step = tf.train.AdagradOptimizer(0.01).minimize(loss)
  4. 修改迭代次数,寻找最高准确率的迭代次数或者使用dropout
  5. 增加神经网络层数或者使用卷积神经网络

代码:

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data
#载入数据集
mnist = input_data.read_data_sets("MNIST_data",one_hot=True)

#每个批次的大小
batch_size = 50
#计算一共有多少个批次
n_batch = mnist.train.num_examples // batch_size  
#定义两个placeholder
x = tf.placeholder(tf.float32,[None,784])#像素28*28
y = tf.placeholder(tf.float32,[None,10])#0-9共10个标签

#使用dropout
keep_prob=tf.placeholder(tf.float32)
lr = tf.Variable(0.001,dtype=tf.float32)

#多层神经网络
W1 = tf.Variable(tf.truncated_normal(([784,500]),stddev=0.1))
b1 = tf.Variable(tf.zeros([500])+0.1)
L1 = tf.nn.tanh(tf.matmul(x,W1)+b1)
L1_drop = tf.nn.dropout(L1,keep_prob)

W2 = tf.Variable(tf.truncated_normal(([500,500]),stddev=0.1))
b2 = tf.Variable(tf.zeros([500])+0.1)
L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2)
L2_drop = tf.nn.dropout(L2,keep_prob)

W3 = tf.Variable(tf.truncated_normal(([500,10]),stddev=0.1))
b3 = tf.Variable(tf.zeros([10])+0.1)

prediction = tf.nn.softmax(tf.matmul(L2_drop,W3)+b3)

#使用释然代价函数
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))
#使用优化
train_step = tf.train.AdamOptimizer(lr).minimize(loss)

#初始化变量
init = tf.global_variables_initializer()
#结果存放在一个布尔型列表中
correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(prediction,1))#argmax返回一维张量中最大的值所在的位置
#求准确率
accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32))#将correct_prediction先转化为浮点型,然后求平均值就是准确率
 
with tf.Session() as sess:
    sess.run(init)
    for epoch in range(31):
        sess.run(tf.assign(lr, 0.001*(0.95**epoch)))
        for batch in range(n_batch):
            batch_xs,batch_ys = mnist.train.next_batch(batch_size)
            sess.run(train_step,feed_dict={x:batch_xs,y:batch_ys,keep_prob:0.8})      
       
        acc = sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels,keep_prob:0.8})
        print("Iter " + str(epoch) + ",Testing Accuracy = " + str(acc))

运行结果:为了方便,只运行了30次已经接近98%的准确率

Iter 0,Testing Accuracy = 0.9335
Iter 1,Testing Accuracy = 0.9502
Iter 2,Testing Accuracy = 0.9576
Iter 3,Testing Accuracy = 0.9611
Iter 4,Testing Accuracy = 0.9671
Iter 5,Testing Accuracy = 0.9663
Iter 6,Testing Accuracy = 0.9648
Iter 7,Testing Accuracy = 0.9677
Iter 8,Testing Accuracy = 0.9675
Iter 9,Testing Accuracy = 0.9685
Iter 10,Testing Accuracy = 0.9687
Iter 11,Testing Accuracy = 0.9718
Iter 12,Testing Accuracy = 0.9738
Iter 13,Testing Accuracy = 0.9738
Iter 14,Testing Accuracy = 0.9764
Iter 15,Testing Accuracy = 0.9754
Iter 16,Testing Accuracy = 0.9762
Iter 17,Testing Accuracy = 0.9755
Iter 18,Testing Accuracy = 0.9748
Iter 19,Testing Accuracy = 0.9757
Iter 20,Testing Accuracy = 0.9746
Iter 21,Testing Accuracy = 0.9763
Iter 22,Testing Accuracy = 0.977
Iter 23,Testing Accuracy = 0.9763
Iter 24,Testing Accuracy = 0.9769
Iter 25,Testing Accuracy = 0.9786
Iter 26,Testing Accuracy = 0.9759
Iter 27,Testing Accuracy = 0.9769
Iter 28,Testing Accuracy = 0.9774
Iter 29,Testing Accuracy = 0.9779
Iter 30,Testing Accuracy = 0.9768
  • 上述程序主要进行了以下优化:
  1. 增加了神经网络层数,增加隐藏层神经网络的数目
    W1 = tf.Variable(tf.truncated_normal(([784,500]),stddev=0.1))
    b1 = tf.Variable(tf.zeros([500])+0.1)
    L1 = tf.nn.tanh(tf.matmul(x,W1)+b1)
    L1_drop = tf.nn.dropout(L1,keep_prob)
    
    W2 = tf.Variable(tf.truncated_normal(([500,500]),stddev=0.1))
    b2 = tf.Variable(tf.zeros([500])+0.1)
    L2 = tf.nn.tanh(tf.matmul(L1_drop,W2)+b2)
    L2_drop = tf.nn.dropout(L2,keep_prob)
    
    W3 = tf.Variable(tf.truncated_normal(([500,10]),stddev=0.1))
    b3 = tf.Variable(tf.zeros([10])+0.1)

    2.使用交叉熵代价函数和优化器,交叉熵一般与softmax回归一起使用,优化器有好多种,例如Adagrad、Rmsporp等

#使用释然代价函数
loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=prediction))
#使用Adam优化
train_step = tf.train.AdamOptimizer(lr).minimize(loss)
  • 新的程序在大的方向主要进行了上述优化,还有其他细节:

1.改变了批次大小: batch_size = 100  (不需要太纠结,这里对准确率的提高作用并不是太大)

2.使用了dropout:keep_prob=tf.placeholder(tf.float32)
dropout可以设置工作的神经网络的百分比,防止过拟合问题,设置为 keep_prob:0.8 表示80%神经网络工作

3.对学习率进行了设置:lr = tf.Variable(0.001,dtype=tf.float32)      sess.run(tf.assign(lr, 0.001*(0.95**epoch)))

可以使学习率发生改变,刚开始没有收敛,学习率比较大,慢慢接近收敛时学习率也减小,这样可以收敛得更快

tensorflow 优化 MNIST数据集分类 Jupyter

喜欢交流的童鞋可以扫描加 qq 一同进步,谢谢啦