Tensorflow 卷积神经网络（更新中）

前言

kears中文文档

* 代表optional，在搭建判断是猫还是狗的卷积神经网络中,不是必备步骤。

Dogs vs. Cats 分类

数据集下载地址

预处理

解压数据

import os
import zipfile

local_zip = '/tmp/cats_and_dogs_filtered.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('/tmp')
zip_ref.close()

定义文件夹

base_dir = '/tmp/cats_and_dogs_filtered'

# 拼接路径
train_dir = os.path.join(base_dir, 'train') 
validation_dir = os.path.join(base_dir, 'validation')

# Directory with our training cat/dog pictures
train_cats_dir = os.path.join(train_dir, 'cats')
train_dogs_dir = os.path.join(train_dir, 'dogs')

# Directory with our validation cat/dog pictures
validation_cats_dir = os.path.join(validation_dir, 'cats')
validation_dogs_dir = os.path.join(validation_dir, 'dogs')

* 打印前10张文件名

train_cat_fnames = os.listdir( train_cats_dir ) #os.listdir() 方法用于返回指定的文件夹包含的文件或文件夹的名字的列表
train_dog_fnames = os.listdir( train_dogs_dir )

print(train_cat_fnames[:10])
print(train_dog_fnames[:10])

"""
打印结果：
['cat.277.jpg', 'cat.424.jpg', 'cat.809.jpg', 'cat.148.jpg', 'cat.114.jpg', 'cat.264.jpg', 'cat.392.jpg', 'cat.355.jpg', 'cat.671.jpg', 'cat.562.jpg']
['dog.928.jpg', 'dog.88.jpg', 'dog.470.jpg', 'dog.782.jpg', 'dog.346.jpg', 'dog.383.jpg', 'dog.728.jpg', 'dog.979.jpg', 'dog.389.jpg', 'dog.71.jpg']
"""

* 打印图片数字

print('total training cat images :', len(os.listdir(      train_cats_dir ) ))
print('total training dog images :', len(os.listdir(      train_dogs_dir ) ))

print('total validation cat images :', len(os.listdir( validation_cats_dir ) ))
print('total validation dog images :', len(os.listdir( validation_dogs_dir ) ))

"""
打印结果：
total training cat images : 1000
total training dog images : 1000
total validation cat images : 500
total validation dog images : 500
"""

* 打印一些猫狗图片

%matplotlib inline

import matplotlib.image as mpimg
import matplotlib.pyplot as plt

# Parameters for our graph; we'll output images in a 4x4 configuration
nrows = 4
ncols = 4

pic_index = 0 # Index for iterating over images


# Set up matplotlib fig, and size it to fit 4x4 pics
fig = plt.gcf()		 # Get Current Figure
fig.set_size_inches(ncols*4, nrows*4)

pic_index+=8

next_cat_pix = [os.path.join(train_cats_dir, fname) 
                for fname in train_cat_fnames[ pic_index-8:pic_index] 
               ]

next_dog_pix = [os.path.join(train_dogs_dir, fname) 
                for fname in train_dog_fnames[ pic_index-8:pic_index]
               ]

for i, img_path in enumerate(next_cat_pix+next_dog_pix):
  # Set up subplot; subplot indices start at 1
  sp = plt.subplot(nrows, ncols, i + 1)
  sp.axis('Off') # Don't show axes (or gridlines)

  img = mpimg.imread(img_path)
  plt.imshow(img)

plt.show()

搭建模型

import tensorflow as tf

model = tf.keras.models.Sequential([
    # Note the input shape is the desired size of the image 150x150 with 3 bytes color
    tf.keras.layers.Conv2D(16, (3,3), activation='relu', input_shape=(150, 150, 3)),
    tf.keras.layers.MaxPooling2D(2,2),
    tf.keras.layers.Conv2D(32, (3,3), activation='relu'),
    tf.keras.layers.MaxPooling2D(2,2), 
    tf.keras.layers.Conv2D(64, (3,3), activation='relu'), 
    tf.keras.layers.MaxPooling2D(2,2),
    # Flatten the results to feed into a DNN
    tf.keras.layers.Flatten(), 
    # 512 neuron hidden layer
    tf.keras.layers.Dense(512, activation='relu'), 
    # Only 1 output neuron. It will contain a value from 0-1 where 0 for 1 class ('cats') and 1 for the other ('dogs')
    tf.keras.layers.Dense(1, activation='sigmoid')  
]
    
model.summary()

打印模型概述信息：

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_6 (Conv2D)            (None, 148, 148, 16)      448       
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 74, 74, 16)        0         
_________________________________________________________________
conv2d_7 (Conv2D)            (None, 72, 72, 32)        4640      
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 36, 36, 32)        0         
_________________________________________________________________
conv2d_8 (Conv2D)            (None, 34, 34, 64)        18496     
_________________________________________________________________
max_pooling2d_8 (MaxPooling2 (None, 17, 17, 64)        0         
_________________________________________________________________
flatten_2 (Flatten)          (None, 18496)             0         
_________________________________________________________________
dense_4 (Dense)              (None, 512)               9470464   
_________________________________________________________________
dense_5 (Dense)              (None, 1)                 513       
=================================================================
Total params: 9,494,561
Trainable params: 9,494,561
Non-trainable params: 0
_________________________________________________________________

编译模型

from tensorflow.keras.optimizers import RMSprop

model.compile(optimizer=RMSprop(lr=0.001),
              loss='binary_crossentropy',
              metrics = ['accuracy'])

数据预处理

创建图片分类器

from tensorflow.keras.preprocessing.image import ImageDataGenerator

# All images will be rescaled by 1./255.
train_datagen = ImageDataGenerator( rescale = 1.0/255. )
test_datagen  = ImageDataGenerator( rescale = 1.0/255. )

# --------------------
# Flow training images in batches of 20 using train_datagen generator
# --------------------
train_generator = train_datagen.flow_from_directory(train_dir,
                                                    batch_size=20,
                                                    class_mode='binary',
                                                    target_size=(150, 150))     
# --------------------
# Flow validation images in batches of 20 using test_datagen generator
# --------------------
validation_generator =  test_datagen.flow_from_directory(validation_dir,
                                                         batch_size=20,
                                                         class_mode  = 'binary',
                                                         target_size = (150, 150))

训练模型

history = model.fit(train_generator,
                    validation_data=validation_generator,
                    steps_per_epoch=100,
                    epochs=15,
                    validation_steps=50,
                    verbose=2)

打印结果：

Epoch 1/15
100/100 - 55s - loss: 0.0300 - accuracy: 0.9925 - val_loss: 1.9040 - val_accuracy: 0.7120
Epoch 2/15
100/100 - 55s - loss: 0.0350 - accuracy: 0.9890 - val_loss: 1.7071 - val_accuracy: 0.7210
Epoch 3/15
100/100 - 55s - loss: 0.0188 - accuracy: 0.9925 - val_loss: 2.0725 - val_accuracy: 0.6950
Epoch 4/15
100/100 - 55s - loss: 0.0291 - accuracy: 0.9915 - val_loss: 2.3252 - val_accuracy: 0.7080
Epoch 5/15
100/100 - 55s - loss: 0.0303 - accuracy: 0.9940 - val_loss: 2.1299 - val_accuracy: 0.7340
Epoch 6/15
100/100 - 55s - loss: 0.0193 - accuracy: 0.9930 - val_loss: 2.0416 - val_accuracy: 0.7400
Epoch 7/15
100/100 - 55s - loss: 0.0197 - accuracy: 0.9925 - val_loss: 2.1636 - val_accuracy: 0.7300
Epoch 8/15
100/100 - 55s - loss: 0.0251 - accuracy: 0.9925 - val_loss: 2.4540 - val_accuracy: 0.7360
Epoch 9/15
100/100 - 55s - loss: 0.0180 - accuracy: 0.9955 - val_loss: 2.7262 - val_accuracy: 0.7260
Epoch 10/15
100/100 - 60s - loss: 0.0163 - accuracy: 0.9960 - val_loss: 2.8085 - val_accuracy: 0.7040
Epoch 11/15
100/100 - 55s - loss: 0.0400 - accuracy: 0.9905 - val_loss: 2.6529 - val_accuracy: 0.7270
Epoch 12/15
100/100 - 55s - loss: 0.0193 - accuracy: 0.9945 - val_loss: 2.8958 - val_accuracy: 0.7220
Epoch 13/15
100/100 - 55s - loss: 0.0204 - accuracy: 0.9950 - val_loss: 3.2313 - val_accuracy: 0.7250
Epoch 14/15
100/100 - 55s - loss: 9.9441e-05 - accuracy: 1.0000 - val_loss: 3.5450 - val_accuracy: 0.7270
Epoch 15/15
100/100 - 55s - loss: 0.0781 - accuracy: 0.9905 - val_loss: 3.2831 - val_accuracy: 0.6940

运行模型

import numpy as np

from google.colab import files
from keras.preprocessing import image

uploaded=files.upload()

for fn in uploaded.keys():
 
  # predicting images
  path='/content/' + fn
  img=image.load_img(path, target_size=(150, 150))
  
  x=image.img_to_array(img)
  x=np.expand_dims(x, axis=0)
  images = np.vstack([x])
  
  classes = model.predict(images, batch_size=10)
  
  print(classes[0])
  
  if classes[0]>0:
    print(fn + " is a dog")
    
  else:
    print(fn + " is a cat")

通过裁剪图片，可以改变分类的结果。

模型准确率及损失评估

#-----------------------------------------------------------
# Retrieve a list of list results on training and test data
# sets for each training epoch
#-----------------------------------------------------------
acc      = history.history[     'accuracy' ]
val_acc  = history.history[ 'val_accuracy' ]
loss     = history.history[    'loss' ]
val_loss = history.history['val_loss' ]

epochs   = range(len(acc)) # Get number of epochs

#------------------------------------------------
# Plot training and validation accuracy per epoch
#------------------------------------------------
plt.plot  ( epochs,     acc )
plt.plot  ( epochs, val_acc )
plt.title ('Training and validation accuracy')
plt.figure()

#------------------------------------------------
# Plot training and validation loss per epoch
#------------------------------------------------
plt.plot  ( epochs,     loss )
plt.plot  ( epochs, val_loss )
plt.title ('Training and validation loss'   )

Training and Validation

总结

**How to build a Dogs vs. Cats classifier? **

• Unzip datas. 解压数据
• Define Directories. 指定文件夹
• Build the model. 搭建模型 （model.Sequential)
• Compile the model. 编译模型
• Create image generator. 创建图像生成器（标准化数据、flow_from_directory读文件）
• Train the model. 训练模型（注意：是model.fit_generator)
• Running the model. （测试新图片）
• Evaluating Accuracy and Loss for the Model. （评估准确性和损失）

增强(Augmentation)

如图：在20个epoches后，训练集的准确性接近1，而验证集的准确性并没有提升，说明此时已经出现了过拟合。

ImageDataGenerator 详情

from tensorflow.keras.preprocessing.image import ImageDataGenerator

train_datagen = ImageDataGenerator(
    rescale=(1./255),
    rotation_range=40,  
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    fill_mode='nearest'
)

• rescale: 重缩放因子。默认为 None。如果是 None 或 0，不进行缩放，否则将数据乘以所提供的值（在应用任何其他转换之前）。

• rotation_range: 整数。随机旋转的度数范围。

• width_shift_range: 浮点数、一维数组或整数

  • float: 如果 <1，则是除以总宽度的值，或者如果 >=1，则为像素值。
  • 1-D 数组: 数组中的随机元素。
  • int: 来自间隔 (-width_shift_range, +width_shift_range) 之间的整数个像素。
  • width_shift_range=2 时，可能值是整数 [-1, 0, +1]，与 width_shift_range=[-1, 0, +1] 相同；而 width_shift_range=1.0 时，可能值是 [-1.0, +1.0) 之间的浮点数。

• height_shift_range: 浮点数、一维数组或整数

  • float: 如果 <1，则是除以总宽度的值，或者如果 >=1，则为像素值。
  • 1-D array-like: 数组中的随机元素。
  • int: 来自间隔 (-height_shift_range, +height_shift_range) 之间的整数个像素。
  • height_shift_range=2 时，可能值是整数 [-1, 0, +1]，与 height_shift_range=[-1, 0, +1] 相同；而 height_shift_range=1.0 时，可能值是 [-1.0, +1.0) 之间的浮点数。

• shear_range: 浮点数。剪切强度（以弧度逆时针方向剪切角度）。

• zoom_range: 浮点数 或 [lower, upper]。随机缩放范围。如果是浮点数，[lower, upper] = [1-zoom_range, 1+zoom_range]。

• horizontal_flip: 布尔值。随机水平翻转。

• fill_mode: {“constant”, “nearest”, “reflect” or “wrap”} 之一。默认为 ‘nearest’。输入边界以外的点根据给定的模式填充：

  • ‘constant’: kkkkkkkk|abcd|kkkkkkkk (cval=k)

  • ‘nearest’: aaaaaaaa|abcd|dddddddd

  • ‘reflect’: abcddcba|abcd|dcbaabcd

  • ‘wrap’: abcdabcd|abcd|abcdabcd

修改了ImageDataGenerator后：

如果只对训练集的ImageDataGenerator做出修改，会使得验证集波动过大，所以要对训练集和验证集的ImageDataGenerator做出同样的修改。

本文地址：https://blog.csdn.net/qq_44815573/article/details/107881368