欢迎您访问程序员文章站本站旨在为大家提供分享程序员计算机编程知识!
您现在的位置是: 首页  >  IT编程

前向传播(张量)- 实战

程序员文章站 2022-03-25 16:16:20
[TOC] 手写数字识别流程 MNIST手写数字集7000 10张图片 60k张图片训练,10k张图片测试 每张图片是28\ 28,如果是彩色图片是28\ 28\ 3 0 255表示图片的灰度值,0表示纯白,255表示纯黑 打平28 28的矩阵,得到28\ 28=784的向量 对于b张图片得到[b, ......

目录

手写数字识别流程

  • mnist手写数字集7000*10张图片
  • 60k张图片训练,10k张图片测试
  • 每张图片是28*28,如果是彩色图片是28*28*3
  • 0-255表示图片的灰度值,0表示纯白,255表示纯黑
  • 打平28*28的矩阵,得到28*28=784的向量
  • 对于b张图片得到[b,784];然后对于b张图片可以给定编码
  • 把上述的普通编码给定成独热编码,但是独热编码都是概率值,并且概率值相加为1,类似于softmax回归
  • 套用线性回归公式
  • x[b,784] w[784,10] b[10] 得到 [b,10]
  • 高维图片实现非常复杂,一个线性模型无法完成,因此可以添加非线性因子
  • f(x@w+b),使用激活函数让其非线性化,引出relu函数
  • 用了激活函数,模型还是太简单
  • 使用工厂
    • h1 =relu(x@w1+b1)
    • h2 = relu(h1@w2+b2)
    • out = relu(h2@w3+b3)
  • 第一步,把[1,784]变成[1,512]变成[1,256]变成[1,10]
  • 得到[1,10]后将结果进行独热编码
  • 使用欧氏距离或者使用mse进行误差度量
  • [1,784]通过三层网络输出一个[1,10]

前向传播(张量)- 实战

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets
import os
# do not print irrelevant information
# os.environ['tf_cpp_min_log_level'] = '2'
# x: [60k,28,28]
# y: [60k]
(x, y), _ = datasets.mnist.load_data()
downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 1s 0us/step
# transform tensor
# x: [0~255] ==》 [0~1.]
x = tf.convert_to_tensor(x, dtype=tf.float32) / 255.
y = tf.convert_to_tensor(y, dtype=tf.int32)
f'x.shape: {x.shape}, y.shape: {y.shape}, x.dtype: {x.dtype}, y.dtype: {y.dtype}'
"x.shape: (60000, 28, 28), y.shape: (60000,), x.dtype: <dtype: 'float32'>, y.dtype: <dtype: 'int32'>"
f'min_x: {tf.reduce_min(x)}, max_x: {tf.reduce_max(x)}'
'min_x: 0.0, max_x: 1.0'
f'min_y: {tf.reduce_min(y)}, max_y: {tf.reduce_max(y)}'
'min_y: 0, max_y: 9'
# batch of 128
train_db = tf.data.dataset.from_tensor_slices((x, y)).batch(128)
train_iter = iter(train_db)
sample = next(train_iter)
f'batch: {sample[0].shape,sample[1].shape}'
'batch: (tensorshape([128, 28, 28]), tensorshape([128]))'
# [b,784] ==> [b,256] ==> [b,128] ==> [b,10]
# [dim_in,dim_out],[dim_out]
w1 = tf.variable(tf.random.truncated_normal([784, 256], stddev=0.1))
b1 = tf.variable(tf.zeros([256]))
w2 = tf.variable(tf.random.truncated_normal([256, 128], stddev=0.1))
b2 = tf.variable(tf.zeros([128]))
w3 = tf.variable(tf.random.truncated_normal([128, 10], stddev=0.1))
b3 = tf.variable(tf.zeros([10]))
# learning rate
lr = 1e-3
for epoch in range(10):  # iterate db for 10
    # tranin every train_db
    for step, (x, y) in enumerate(train_db):
        # x: [128,28,28]
        # y: [128]

        # [b,28,28] ==> [b,28*28]
        x = tf.reshape(x, [-1, 28*28])

        with tf.gradienttape() as tape:  # only data types of tf.variable are logged
            # x: [b,28*28]
            # h1 = x@w1 + b1
            # [b,784]@[784,256]+[256] ==> [b,256] + [256] ==> [b,256] + [b,256]
            h1 = x @ w1 + tf.broadcast_to(b1, [x.shape[0], 256])
            h1 = tf.nn.relu(h1)
            # [b,256] ==> [b,128]
            # h2 = x@w2 + b2  # b2 can broadcast automatic
            h2 = h1 @ w2 + b2
            h2 = tf.nn.relu(h2)
            # [b,128] ==> [b,10]
            out = h2 @ w3 + b3

            # compute loss
            # out: [b,10]
            # y:[b] ==> [b,10]
            y_onehot = tf.one_hot(y, depth=10)

            # mse = mean(sum(y-out)^2)
            # [b,10]
            loss = tf.square(y_onehot - out)
            # mean:scalar
            loss = tf.reduce_mean(loss)

        # compute gradients
        grads = tape.gradient(loss, [w1, b1, w2, b2, w3, b3])
        # w1 = w1 - lr * w1_grad
        # w1 = w1 - lr * grads[0]  # not in situ update
        # in situ update
        w1.assign_sub(lr * grads[0])
        b1.assign_sub(lr * grads[1])
        w2.assign_sub(lr * grads[2])
        b2.assign_sub(lr * grads[3])
        w3.assign_sub(lr * grads[4])
        b3.assign_sub(lr * grads[5])

        if step % 100 == 0:
            print(f'epoch:{epoch}, step: {step}, loss:{float(loss)}')
epoch:0, step: 0, loss:0.5366693735122681
epoch:0, step: 100, loss:0.23276552557945251
epoch:0, step: 200, loss:0.19647717475891113
epoch:0, step: 300, loss:0.17389704287052155
epoch:0, step: 400, loss:0.1731622964143753
epoch:1, step: 0, loss:0.16157487034797668
epoch:1, step: 100, loss:0.16654588282108307
epoch:1, step: 200, loss:0.15311869978904724
epoch:1, step: 300, loss:0.14135733246803284
epoch:1, step: 400, loss:0.14423415064811707
epoch:2, step: 0, loss:0.13703864812850952
epoch:2, step: 100, loss:0.14255204796791077
epoch:2, step: 200, loss:0.1302051544189453
epoch:2, step: 300, loss:0.12224273383617401
epoch:2, step: 400, loss:0.12742099165916443
epoch:3, step: 0, loss:0.1219201311469078
epoch:3, step: 100, loss:0.12757658958435059
epoch:3, step: 200, loss:0.11587800830602646
epoch:3, step: 300, loss:0.10984969139099121
epoch:3, step: 400, loss:0.11641304194927216
epoch:4, step: 0, loss:0.11171815544366837
epoch:4, step: 100, loss:0.11717887222766876
epoch:4, step: 200, loss:0.10604140907526016
epoch:4, step: 300, loss:0.10111508518457413
epoch:4, step: 400, loss:0.10865814983844757
epoch:5, step: 0, loss:0.10434548556804657
epoch:5, step: 100, loss:0.10952303558588028
epoch:5, step: 200, loss:0.09875871241092682
epoch:5, step: 300, loss:0.09467941522598267
epoch:5, step: 400, loss:0.10282392799854279
epoch:6, step: 0, loss:0.09874211996793747
epoch:6, step: 100, loss:0.10355912148952484
epoch:6, step: 200, loss:0.09315416216850281
epoch:6, step: 300, loss:0.08971598744392395
epoch:6, step: 400, loss:0.0982089415192604
epoch:7, step: 0, loss:0.09428335726261139
epoch:7, step: 100, loss:0.09877124428749084
epoch:7, step: 200, loss:0.08866965025663376
epoch:7, step: 300, loss:0.08573523908853531
epoch:7, step: 400, loss:0.09440126270055771
epoch:8, step: 0, loss:0.09056715667247772
epoch:8, step: 100, loss:0.09483197331428528
epoch:8, step: 200, loss:0.0849832147359848
epoch:8, step: 300, loss:0.08246967941522598
epoch:8, step: 400, loss:0.09117519855499268
epoch:9, step: 0, loss:0.08741479367017746
epoch:9, step: 100, loss:0.09150294959545135
epoch:9, step: 200, loss:0.08185736835002899
epoch:9, step: 300, loss:0.07972464710474014
epoch:9, step: 400, loss:0.08842341601848602