pytorch：neural network

参考链接：
https://pytorch.org/tutorials/beginner/blitz/neural_networks_tutorial.html#sphx-glr-beginner-blitz-neural-networks-tutorial-py

一个典型的神经网络的训练过程可以描述如下：

1. 定义神经网络（包含一些可以训练的参数）；
2. 根据输入的数据集进行迭代；
3. 通过网络架构处理输入；
4. 计算损失函数；
5. 传播梯度给网络的参数；
6. 更新网络的权重，一般使用weight = weight - learning_rate * gradient

import torch 
import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)
    
    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))
        x = F.max_pool2d(F.relu(self.conv2(x)), 2)
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))   
        x = self.fc3(x)
        return x
    
    def num_flat_features(self, x):
        size = x.size()[1:]
        num_feature = 1
        for s in size:
            num_feature *= s
        return num_feature
    
net = Net()
print(net)
# 网络模型的参数
params = list(net.parameters())
print(len(params))        # 10
print(params[0].size())   # (6, 1, 5, 5)

input = torch.randn(1, 1, 32, 32)
out = net(input)
print(out)

output = net(input)
target = torch.randn(10)
target = target.view(1, -1)
criterion = nn.MSELoss()

loss = criterion(output, target)
print(loss)
print(loss.grad_fn) # MSELoss
print(loss.grad_fn.next_functions[0][0]) # Linear
print(loss.grad_fn.next_functions[0][0].next_functions[0][0]) # ReLU

整个网络的计算过程为：
input
conv2d relu maxpool2d conv2d relu maxpool2d view linear relu linear relu linear relu linear MSELoss
loss
torch.nn只支持mini-batches的输入，但是不支持单个样本作为输入，对于单样本的时候，可以使用input.unsqueeze(0)来冒充一个batch维度的输入

更新参数：

learning_rate = 0.01
for f in net.parameters():
    f.data.sub_(f.grad.data * learning_rate)

优化

import torch.optim as optim

# create your optimizer
optimizer = optim.SGD(net.parameters(), lr=0.01)

# in your training loop:
optimizer.zero_grad()   # zero the gradient buffers
output = net(input)
loss = criterion(output, target)
loss.backward()
optimizer.step()