ResNet解析

程序员文章站 2022-07-01 17:36:20

...

转载出处：https://blog.csdn.net/lanran2/article/details/79057994#commentBox
论文出处：https://arxiv.org/abs/1512.03385
ResNet在2015年被提出，在ImageNet比赛classification任务上获得第一名，因为它“简单与实用”并存，之后很多方法都建立在ResNet50或者ResNet101的基础上完成的，检测，分割，识别等领域都纷纷使用ResNet，Alpha zero也使用了ResNet，所以可见ResNet确实很好用。
下面我们从实用的角度去看看ResNet。

1.ResNet意义

随着网络的加深，出现了训练集准确率下降的现象，我们可以确定这不是由于Overfit过拟合造成的(过拟合的情况训练集应该准确率很高)；所以作者针对这个问题提出了一种全新的网络，叫深度残差网络，它允许网络尽可能的加深，其中引入了全新的结构如图1；
这里问大家一个问题
残差指的是什么？
其中ResNet提出了两种mapping：一种是identity mapping，指的就是图1中”弯弯的曲线”，另一种residual mapping，指的就是除了”弯弯的曲线“那部分，所以最后的输出是 y=F(x)+xy=F(x)+x
identity mapping顾名思义，就是指本身，也就是公式中的xx，而residual mapping指的是“差”，也就是y−xy−x，所以残差指的就是F(x)F(x)部分。
为什么ResNet可以解决“随着网络加深，准确率不下降”的问题？
除了实验证明外：
ResNet解析
这里写图片描述
表1，Resnet在ImageNet上的结果
理论上，对于“随着网络加深，准确率下降”的问题，Resnet提供了两种选择方式，也就是identity mapping和residual mapping，如果网络已经到达最优，继续加深网络，residual mapping将被push为0，只剩下identity mapping，这样理论上网络一直处于最优状态了，网络的性能也就不会随着深度增加而降低了。

2.ResNet结构

它使用了一种连接方式叫做“shortcut connection”，顾名思义，shortcut就是“抄近道”的意思，看下图我们就能大致理解：
ResNet解析
这里写图片描述
图1 Shortcut Connection
这是文章里面的图，我们可以看到一个“弯弯的弧线“这个就是所谓的”shortcut connection“，也是文中提到identity mapping，这张图也诠释了ResNet的真谛，当然大家可以放心，真正在使用的ResNet模块并不是这么单一，文章中就提出了两种方式：
ResNet解析
这里写图片描述
图2 两种ResNet设计
这两种结构分别针对ResNet34（左图）和ResNet50/101/152（右图），一般称整个结构为一个”building block“。其中右图又称为”bottleneck design”，目的一目了然，就是为了降低参数的数目，第一个1x1的卷积把256维channel降到64维，然后在最后通过1x1卷积恢复，整体上用的参数数目：1x1x256x64 + 3x3x64x64 + 1x1x64x256 = 69632，而不使用bottleneck的话就是两个3x3x256的卷积，参数数目: 3x3x256x256x2 = 1179648，差了16.94倍。
对于常规ResNet，可以用于34层或者更少的网络中，对于Bottleneck Design的ResNet通常用于更深的如101这样的网络中，目的是减少计算和参数量（实用目的）。

问大家一个问题：
如图1所示，如果F(x)和x的channel个数不同怎么办，因为F(x)和x是按照channel维度相加的，channel不同怎么相加呢？
针对channel个数是否相同，要分成两种情况考虑，如下图：
ResNet解析这里写图片描述
图3 两种Shortcut Connection方式
如图3所示，我们可以清楚的”实线“和”虚线“两种连接方式，
实线的的Connection部分(”第一个粉色矩形和第三个粉色矩形“)都是执行3x3x64的卷积，他们的channel个数一致，所以采用计算方式：
y=F(x)+xy=F(x)+x
虚线的的Connection部分(”第一个绿色矩形和第三个绿色矩形“)分别是3x3x64和3x3x128的卷积操作，他们的channel个数不同(64和128)，所以采用计算方式：
y=F(x)+Wxy=F(x)+Wx
其中W是卷积操作，用来调整x的channel维度的；
下面我们看看两个实例：
ResNet解析
这里写图片描述
图4 两种Shortcut Connection方式实例（左图channel一致，右图channel不一样）

3.ResNet50和ResNet101

这里把ResNet50和ResNet101特别提出，主要因为它们的出镜率很高，所以需要做特别的说明。给出了它们具体的结构：
这里写图片描述
ResNet解析
表2，Resnet不同的结构
首先我们看一下表2，上面一共提出了5中深度的ResNet，分别是18，34，50，101和152，首先看表2最左侧，我们发现所有的网络都分成5部分，分别是：conv1，conv2_x，conv3_x，conv4_x，conv5_x，之后的其他论文也会专门用这个称呼指代ResNet50或者101的每部分。
拿101-layer那列，我们先看看101-layer是不是真的是101层网络，首先有个输入7x7x64的卷积，然后经过3 + 4 + 23 + 3 = 33个building block，每个block为3层，所以有33 x 3 = 99层，最后有个fc层(用于分类)，所以1 + 99 + 1 = 101层，确实有101层网络；
注：101层网络仅仅指卷积或者全连接层，而**层或者Pooling层并没有计算在内；
这里我们关注50-layer和101-layer这两列，可以发现，它们唯一的不同在于conv4_x，ResNet50有6个block，而ResNet101有23个block，查了17个block，也就是17 x 3 = 51层。

4.基于ResNet101的Faster RCNN

文章中把ResNet101应用在Faster RCNN上取得了更好的结果，结果如下：
ResNet解析
这里写图片描述
这里写图片描述
表3，Resnet101 Faster RCNN在Pascal VOC07/12 以及COCO上的结果
这里有个问题：
Faster RCNN中RPN和Fast RCNN的共享特征图用的是conv5_x的输出么？
针对这个问题我们看看实际的基于ResNet101的Faster RCNN的结构图：
ResNet解析
这里写图片描述
图5 基于ResNet101的Faster RCNN
图5展示了整个Faster RCNN的架构，其中蓝色的部分为ResNet101，可以发现conv4_x的最后的输出为RPN和RoI Pooling共享的部分，而conv5_x(共9层网络)都作用于RoI Pooling之后的一堆特征图(14 x 14 x 1024)，特征图的大小维度也刚好符合原本的ResNet101中conv5_x的输入；
最后大家一定要记得最后要接一个average pooling，得到2048维特征，分别用于分类和框回归。
以下为caffe中ResNet34中的proto.txt
只需要将其复制粘贴到Netscope便可以查看ResNet34每一层的所有参数，加深对ResNet34结构的总体认知。

name: "ResNet-34"
layer {
    name: "data"
    type: "Data"
    top: "data"
    top: "label"
    include {
        phase: TRAIN
    }
    transform_param {
        mirror: true
        crop_size: 224
        mean_value: 104
        mean_value: 117
        mean_value: 123
    }
    data_param {
        source: "../ilsvrc2012/ilsvrc2012_train"
        batch_size: 32
        backend: LMDB
    }
}
layer {
    name: "data"
    type: "Data"
    top: "data"
    top: "label"
    include {
        phase: TEST
    }
    transform_param {
        mirror: false
        crop_size: 224
        mean_value: 104
        mean_value: 117
        mean_value: 123
    }
    data_param {
        source: "../ilsvrc2012/ilsvrc2012_val"
        batch_size: 1
        backend: LMDB
    }
}

layer {
    bottom: "data"
    top: "conv1"
    name: "conv1"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 7
        pad: 3
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "conv1"
    top: "conv1"
    name: "bn_conv1"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "conv1"
    top: "conv1"
    name: "scale_conv1"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "conv1"
    top: "conv1"
    name: "conv1_relu"
    type: "ReLU"
}

layer {
    bottom: "conv1"
    top: "pool1"
    name: "pool1"
    type: "Pooling"
    pooling_param {
        kernel_size: 3
        stride: 2
        pool: MAX
    }
}

layer {
    bottom: "pool1"
    top: "res2a_branch1"
    name: "res2a_branch1"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 1
        pad: 0
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2a_branch1"
    top: "res2a_branch1"
    name: "bn2a_branch1"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2a_branch1"
    top: "res2a_branch1"
    name: "scale2a_branch1"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "pool1"
    top: "res2a_branch2a"
    name: "res2a_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2a_branch2a"
    top: "res2a_branch2a"
    name: "bn2a_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2a_branch2a"
    top: "res2a_branch2a"
    name: "scale2a_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2a_branch2a"
    top: "res2a_branch2a"
    name: "res2a_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res2a_branch2a"
    top: "res2a_branch2b"
    name: "res2a_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2a_branch2b"
    top: "res2a_branch2b"
    name: "bn2a_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2a_branch2b"
    top: "res2a_branch2b"
    name: "scale2a_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2a_branch1"
    bottom: "res2a_branch2b"
    top: "res2a"
    name: "res2a"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res2a"
    top: "res2a"
    name: "res2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res2a"
    top: "res2b_branch2a"
    name: "res2b_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2b_branch2a"
    top: "res2b_branch2a"
    name: "bn2b_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2b_branch2a"
    top: "res2b_branch2a"
    name: "scale2b_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2b_branch2a"
    top: "res2b_branch2a"
    name: "res2b_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res2b_branch2a"
    top: "res2b_branch2b"
    name: "res2b_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2b_branch2b"
    top: "res2b_branch2b"
    name: "bn2b_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2b_branch2b"
    top: "res2b_branch2b"
    name: "scale2b_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2a"
    bottom: "res2b_branch2b"
    top: "res2b"
    name: "res2b"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res2b"
    top: "res2b"
    name: "res2b_relu"
    type: "ReLU"
}

layer {
    bottom: "res2b"
    top: "res2c_branch2a"
    name: "res2c_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2c_branch2a"
    top: "res2c_branch2a"
    name: "bn2c_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2c_branch2a"
    top: "res2c_branch2a"
    name: "scale2c_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2c_branch2a"
    top: "res2c_branch2a"
    name: "res2c_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res2c_branch2a"
    top: "res2c_branch2b"
    name: "res2c_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 64
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res2c_branch2b"
    top: "res2c_branch2b"
    name: "bn2c_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res2c_branch2b"
    top: "res2c_branch2b"
    name: "scale2c_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2b"
    bottom: "res2c_branch2b"
    top: "res2c"
    name: "res2c"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res2c"
    top: "res2c"
    name: "res2c_relu"
    type: "ReLU"
}

layer {
    bottom: "res2c"
    top: "res3a_branch1"
    name: "res3a_branch1"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 1
        pad: 0
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3a_branch1"
    top: "res3a_branch1"
    name: "bn3a_branch1"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3a_branch1"
    top: "res3a_branch1"
    name: "scale3a_branch1"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res2c"
    top: "res3a_branch2a"
    name: "res3a_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3a_branch2a"
    top: "res3a_branch2a"
    name: "bn3a_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3a_branch2a"
    top: "res3a_branch2a"
    name: "scale3a_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3a_branch2a"
    top: "res3a_branch2a"
    name: "res3a_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res3a_branch2a"
    top: "res3a_branch2b"
    name: "res3a_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3a_branch2b"
    top: "res3a_branch2b"
    name: "bn3a_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3a_branch2b"
    top: "res3a_branch2b"
    name: "scale3a_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3a_branch1"
    bottom: "res3a_branch2b"
    top: "res3a"
    name: "res3a"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res3a"
    top: "res3a"
    name: "res3a_relu"
    type: "ReLU"
}

layer {
    bottom: "res3a"
    top: "res3b_branch2a"
    name: "res3b_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3b_branch2a"
    top: "res3b_branch2a"
    name: "bn3b_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3b_branch2a"
    top: "res3b_branch2a"
    name: "scale3b_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3b_branch2a"
    top: "res3b_branch2a"
    name: "res3b_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res3b_branch2a"
    top: "res3b_branch2b"
    name: "res3b_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3b_branch2b"
    top: "res3b_branch2b"
    name: "bn3b_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3b_branch2b"
    top: "res3b_branch2b"
    name: "scale3b_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3a"
    bottom: "res3b_branch2b"
    top: "res3b"
    name: "res3b"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res3b"
    top: "res3b"
    name: "res3b_relu"
    type: "ReLU"
}

layer {
    bottom: "res3b"
    top: "res3c_branch2a"
    name: "res3c_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3c_branch2a"
    top: "res3c_branch2a"
    name: "bn3c_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3c_branch2a"
    top: "res3c_branch2a"
    name: "scale3c_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3c_branch2a"
    top: "res3c_branch2a"
    name: "res3c_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res3c_branch2a"
    top: "res3c_branch2b"
    name: "res3c_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3c_branch2b"
    top: "res3c_branch2b"
    name: "bn3c_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3c_branch2b"
    top: "res3c_branch2b"
    name: "scale3c_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3b"
    bottom: "res3c_branch2b"
    top: "res3c"
    name: "res3c"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res3c"
    top: "res3c"
    name: "res3c_relu"
    type: "ReLU"
}

layer {
    bottom: "res3c"
    top: "res3d_branch2a"
    name: "res3d_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3d_branch2a"
    top: "res3d_branch2a"
    name: "bn3d_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3d_branch2a"
    top: "res3d_branch2a"
    name: "scale3d_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3d_branch2a"
    top: "res3d_branch2a"
    name: "res3d_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res3d_branch2a"
    top: "res3d_branch2b"
    name: "res3d_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 128
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res3d_branch2b"
    top: "res3d_branch2b"
    name: "bn3d_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res3d_branch2b"
    top: "res3d_branch2b"
    name: "scale3d_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3c"
    bottom: "res3d_branch2b"
    top: "res3d"
    name: "res3d"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res3d"
    top: "res3d"
    name: "res3d_relu"
    type: "ReLU"
}

layer {
    bottom: "res3d"
    top: "res4a_branch1"
    name: "res4a_branch1"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 1
        pad: 0
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4a_branch1"
    top: "res4a_branch1"
    name: "bn4a_branch1"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4a_branch1"
    top: "res4a_branch1"
    name: "scale4a_branch1"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res3d"
    top: "res4a_branch2a"
    name: "res4a_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4a_branch2a"
    top: "res4a_branch2a"
    name: "bn4a_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4a_branch2a"
    top: "res4a_branch2a"
    name: "scale4a_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4a_branch2a"
    top: "res4a_branch2a"
    name: "res4a_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4a_branch2a"
    top: "res4a_branch2b"
    name: "res4a_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4a_branch2b"
    top: "res4a_branch2b"
    name: "bn4a_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4a_branch2b"
    top: "res4a_branch2b"
    name: "scale4a_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4a_branch1"
    bottom: "res4a_branch2b"
    top: "res4a"
    name: "res4a"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4a"
    top: "res4a"
    name: "res4a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4a"
    top: "res4b_branch2a"
    name: "res4b_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4b_branch2a"
    top: "res4b_branch2a"
    name: "bn4b_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4b_branch2a"
    top: "res4b_branch2a"
    name: "scale4b_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4b_branch2a"
    top: "res4b_branch2a"
    name: "res4b_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4b_branch2a"
    top: "res4b_branch2b"
    name: "res4b_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4b_branch2b"
    top: "res4b_branch2b"
    name: "bn4b_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4b_branch2b"
    top: "res4b_branch2b"
    name: "scale4b_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4a"
    bottom: "res4b_branch2b"
    top: "res4b"
    name: "res4b"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4b"
    top: "res4b"
    name: "res4b_relu"
    type: "ReLU"
}

layer {
    bottom: "res4b"
    top: "res4c_branch2a"
    name: "res4c_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4c_branch2a"
    top: "res4c_branch2a"
    name: "bn4c_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4c_branch2a"
    top: "res4c_branch2a"
    name: "scale4c_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4c_branch2a"
    top: "res4c_branch2a"
    name: "res4c_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4c_branch2a"
    top: "res4c_branch2b"
    name: "res4c_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4c_branch2b"
    top: "res4c_branch2b"
    name: "bn4c_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4c_branch2b"
    top: "res4c_branch2b"
    name: "scale4c_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4b"
    bottom: "res4c_branch2b"
    top: "res4c"
    name: "res4c"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4c"
    top: "res4c"
    name: "res4c_relu"
    type: "ReLU"
}

layer {
    bottom: "res4c"
    top: "res4d_branch2a"
    name: "res4d_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4d_branch2a"
    top: "res4d_branch2a"
    name: "bn4d_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4d_branch2a"
    top: "res4d_branch2a"
    name: "scale4d_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4d_branch2a"
    top: "res4d_branch2a"
    name: "res4d_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4d_branch2a"
    top: "res4d_branch2b"
    name: "res4d_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4d_branch2b"
    top: "res4d_branch2b"
    name: "bn4d_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4d_branch2b"
    top: "res4d_branch2b"
    name: "scale4d_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4c"
    bottom: "res4d_branch2b"
    top: "res4d"
    name: "res4d"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4d"
    top: "res4d"
    name: "res4d_relu"
    type: "ReLU"
}

layer {
    bottom: "res4d"
    top: "res4e_branch2a"
    name: "res4e_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4e_branch2a"
    top: "res4e_branch2a"
    name: "bn4e_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4e_branch2a"
    top: "res4e_branch2a"
    name: "scale4e_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4e_branch2a"
    top: "res4e_branch2a"
    name: "res4e_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4e_branch2a"
    top: "res4e_branch2b"
    name: "res4e_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4e_branch2b"
    top: "res4e_branch2b"
    name: "bn4e_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4e_branch2b"
    top: "res4e_branch2b"
    name: "scale4e_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4d"
    bottom: "res4e_branch2b"
    top: "res4e"
    name: "res4e"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4e"
    top: "res4e"
    name: "res4e_relu"
    type: "ReLU"
}

layer {
    bottom: "res4e"
    top: "res4f_branch2a"
    name: "res4f_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4f_branch2a"
    top: "res4f_branch2a"
    name: "bn4f_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4f_branch2a"
    top: "res4f_branch2a"
    name: "scale4f_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4f_branch2a"
    top: "res4f_branch2a"
    name: "res4f_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res4f_branch2a"
    top: "res4f_branch2b"
    name: "res4f_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 256
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res4f_branch2b"
    top: "res4f_branch2b"
    name: "bn4f_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res4f_branch2b"
    top: "res4f_branch2b"
    name: "scale4f_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4e"
    bottom: "res4f_branch2b"
    top: "res4f"
    name: "res4f"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res4f"
    top: "res4f"
    name: "res4f_relu"
    type: "ReLU"
}

layer {
    bottom: "res4f"
    top: "res5a_branch1"
    name: "res5a_branch1"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 1
        pad: 0
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5a_branch1"
    top: "res5a_branch1"
    name: "bn5a_branch1"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5a_branch1"
    top: "res5a_branch1"
    name: "scale5a_branch1"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res4f"
    top: "res5a_branch2a"
    name: "res5a_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 2
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5a_branch2a"
    top: "res5a_branch2a"
    name: "bn5a_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5a_branch2a"
    top: "res5a_branch2a"
    name: "scale5a_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5a_branch2a"
    top: "res5a_branch2a"
    name: "res5a_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res5a_branch2a"
    top: "res5a_branch2b"
    name: "res5a_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5a_branch2b"
    top: "res5a_branch2b"
    name: "bn5a_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5a_branch2b"
    top: "res5a_branch2b"
    name: "scale5a_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5a_branch1"
    bottom: "res5a_branch2b"
    top: "res5a"
    name: "res5a"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res5a"
    top: "res5a"
    name: "res5a_relu"
    type: "ReLU"
}

layer {
    bottom: "res5a"
    top: "res5b_branch2a"
    name: "res5b_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5b_branch2a"
    top: "res5b_branch2a"
    name: "bn5b_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5b_branch2a"
    top: "res5b_branch2a"
    name: "scale5b_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5b_branch2a"
    top: "res5b_branch2a"
    name: "res5b_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res5b_branch2a"
    top: "res5b_branch2b"
    name: "res5b_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5b_branch2b"
    top: "res5b_branch2b"
    name: "bn5b_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5b_branch2b"
    top: "res5b_branch2b"
    name: "scale5b_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5a"
    bottom: "res5b_branch2b"
    top: "res5b"
    name: "res5b"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res5b"
    top: "res5b"
    name: "res5b_relu"
    type: "ReLU"
}

layer {
    bottom: "res5b"
    top: "res5c_branch2a"
    name: "res5c_branch2a"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5c_branch2a"
    top: "res5c_branch2a"
    name: "bn5c_branch2a"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5c_branch2a"
    top: "res5c_branch2a"
    name: "scale5c_branch2a"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5c_branch2a"
    top: "res5c_branch2a"
    name: "res5c_branch2a_relu"
    type: "ReLU"
}

layer {
    bottom: "res5c_branch2a"
    top: "res5c_branch2b"
    name: "res5c_branch2b"
    type: "Convolution"
    convolution_param {
        num_output: 512
        kernel_size: 3
        pad: 1
        stride: 1
        weight_filler {
            type: "msra"
        }
        bias_term: false

    }
}

layer {
    bottom: "res5c_branch2b"
    top: "res5c_branch2b"
    name: "bn5c_branch2b"
    type: "BatchNorm"
    batch_norm_param {
        use_global_stats: false
    }
}

layer {
    bottom: "res5c_branch2b"
    top: "res5c_branch2b"
    name: "scale5c_branch2b"
    type: "Scale"
    scale_param {
        bias_term: true
    }
}

layer {
    bottom: "res5b"
    bottom: "res5c_branch2b"
    top: "res5c"
    name: "res5c"
    type: "Eltwise"
    eltwise_param {
        operation: SUM
    }
}

layer {
    bottom: "res5c"
    top: "res5c"
    name: "res5c_relu"
    type: "ReLU"
}

layer {
    bottom: "res5c"
    top: "pool5"
    name: "pool5"
    type: "Pooling"
    pooling_param {
        kernel_size: 7
        stride: 1
        pool: AVE
    }
}

layer {
    bottom: "pool5"
    top: "fc1000"
    name: "fc1000"
    type: "InnerProduct"
    param {
        lr_mult: 1
        decay_mult: 1
    }
    param {
        lr_mult: 2
        decay_mult: 1
    }
    inner_product_param {
        num_output: 1000
        weight_filler {
            type: "xavier"
        }
        bias_filler {
            type: "constant"
            value: 0
        }
    }
}

layer {
    bottom: "fc1000"
    bottom: "label"
    name: "loss"
    type: "SoftmaxWithLoss"
    top: "loss"
}

layer {
    bottom: "fc1000"
    bottom: "label"
    top: "acc/top-1"
    name: "acc/top-1"
    type: "Accuracy"
    include {
        phase: TEST
    }
}

layer {
    bottom: "fc1000"
    bottom: "label"
    top: "acc/top-5"
    name: "acc/top-5"
    type: "Accuracy"
    include {
        phase: TEST
    }
    accuracy_param {
        top_k: 5
    }
}

ResNet解析

解析php框架codeigniter中如何使用框架的session_PHP教程

Oracle row_number() over()解析函数高效实现分页_Oracle应用_脚

PHP：使用Apache如何在本地搭建泛解析呢？

XML 解析错误：格式不佳？求解

PHP文件锁定写入实例解析

简单解析PHP程序的运行流程，解析php流程

剑指Offer系列（java版，详细解析）53.在排序数组中查找数字

JavaScript 语句解析

多就是好吗？解析智能手机多摄发展困局

解析mysql二进制日志处理事务与非事务性语句的区别_MySQL