防止过拟合-Dropout2d

Dropout的过程
1）按照概率p，对每个输入channel进行伯努利采样，随机采样到的channel置为0，输出
2）将1）的输出结果乘以1/(1-p)就是做了dropout的结果

代码验证：

#%%
import torch
import torch.nn as nn

#%% 模型
conv1 = nn.Conv2d(2,2,kernel_size=3,stride=1,padding=0)
m = nn.Dropout2d(p=0.4)

#%% 数据准备
N = 2
C = 2
H = 4
W = 4
input = torch.arange(N*C*H*W,dtype=torch.float32).view([N,C,H,W])

'''
1）按照概率p，对每个输入channel进行伯努利采样，随机采样到的channel置为0，输出
2）将1）的输出结果乘以1/(1-p)就是做了dropout的结果
'''

#%% 预测 没有做dropout
m.eval()
conv1_out = conv1(input)
dropout_out =  m(conv1_out)
# print('conv1:',conv1_out)
print('没有做dropout:',dropout_out)


#%% 训练
m.train()
conv1_out = conv1(input)
dropout_out =  m(conv1_out)

# print('conv1:',conv1_out)
print('做了dropout:',dropout_out)

结果：

没有做dropout: tensor([[[[-10.0299, -10.0603],
          [-10.1516, -10.1820]],

         [[  5.0803,   5.3178],
          [  6.0304,   6.2679]]],


        [[[-11.0039, -11.0343],
          [-11.1256, -11.1560]],

         [[ 12.6810,  12.9185],
          [ 13.6311,  13.8686]]]], grad_fn=<ThnnConv2DBackward>)

做了dropout: tensor([[[[-16.7164, -16.7672],
          [-16.9193, -16.9701]],

         [[  8.4672,   8.8631],
          [ 10.0507,  10.4465]]],


        [[[-18.3398, -18.3905],
          [-18.5427, -18.5934]],

         [[  0.0000,   0.0000],
          [  0.0000,   0.0000]]]], grad_fn=<MulBackward0>)