残差网络的基本原理

上图即为残差网络的基本原理：

F(x)=H(x)-x，x为浅层输出，H(x)为深层的输出，F(x)为二者中间的两层变换。当浅层的x代表的特征已经足够成熟，如果任何对于x的改变都会让损失变大的话，F(x)会自动趋向于学习为0，x则从恒等映射的路径继续传递。这样在不增加计算成本的情况下实现了这一目的：当浅层的输出已经足够成熟，让深层网络后面的层能够实现恒等映射的作用。

残差网络的基本模块实现

上图中是残差网络中两个最基本的模块，论文中分别命名为“buiding block”和"bottleneck"，分别应用于浅层(18和34)残差网络和深层(50、101和152层)残差网络中。

首先定义3*3卷积操作

def conv3x3(in_planes, out_planes, stride=1): return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride, padding=1, bias=False)

building block的pytorch实现：

class BasicBlock(nn.Module): expansion = 1 def __init__(self, inplanes, planes, stride=1, downsample=None): super(BasicBlock, self).__init__() self.conv1 = conv3x3(inplanes, planes, stride) self.bn1 = nn.BatchNorm2d(planes) self.relu = nn.ReLU(inplace=True) self.conv2 = conv3x3(planes, planes) self.bn2 = nn.BatchNorm2d(planes) self.downsample = downsample self.stride = stride def forward(self, x): residual=x out=self.conv1(x) out=self.bn1(out) out=self.relu(out) out=self.bn2(out) out=self.bn2(out) if self.downsample is not None: residual=self.downsample(x) out+=residual out=self.relu(out) return out

bottleneck的pytorch实现：

class Bottleneck(nn.Module): expansion = 4 def __init__(self, inplanes, planes, stride=1, downsample=None): super(Bottleneck, self).__init__() self.conv1 = nn.Conv2d(inplanes, planes, kernel_size=1, stride=stride, bias=False) self.bn1 = nn.BatchNorm2d(planes) self.conv2 = nn.Conv2d(planes, planes, kernel_size=3, stride=1, padding=1, bias=False) self.bn2 = nn.BatchNorm2d(planes) self.conv3 = nn.Conv2d(planes, planes * 4, kernel_size=1, bias=False) self.bn3 = nn.BatchNorm2d(planes * 4) self.relu = nn.ReLU(inplace=True) self.downsample = downsample self.stride = stride def forward(self, x): residual = x out = self.conv1(x) out = self.bn1(out) out = self.relu(out) out = self.conv2(out) out = self.bn2(out) out = self.relu(out) out = self.conv3(out) out = self.bn3(out) if self.downsample is not None: residual = self.downsample(x) out += residual out = self.relu(out) return out

block模块：具有2个卷积层，均为3*3卷积，后一个卷积层输入和输出的通道数相同

bottleneck模块：具有3个卷积层，第1和第3个均为1*1卷积，用于改变通道数；第二个为3*3卷积，该层输入和输出的通道数相同；第三个卷积层输出的通道数为输入的4倍。

在两个模块中，前面的卷积层的输出后进行都直接加1个relu层，只有最后一个卷积层和原输入x相加后再加一个relu层。

在以上实现中有两个需要注意的问题

1. 在每个模块中，均有一个downsample操作，这是为了防止x和F(x)相加中通道数目不匹配所采取1*1的卷积操作。若x和F(x)通道数目一致，则downsample是空操作(None)，如不一致，则为1*1的卷积操作。
2. 在两个模块中均有一个expansion变量(block模块中值为1，bottleneck模块中值为4)。模块输出的通道数目为planes*expansion。

残差网络的整体架构

图中是论文中给出的resnet-34。整个残差网络可分为：

1. 一个7*7卷积层，该层输入通道数是3(原图的RGB三通道)，输出通道数是64，同时stride=2，即分辨率下降一半；
2. 一个最大值池化层，stride=2，同样分辨率下降一半；
3. 残差层，可分为4个stage，同一stage内通道数目相同。在上图中，出现虚线时表明，进入了下一个stage。虚线模块完成了分辨率减少1/2(卷积时stride=2)，同时通道数目增加1倍的操作，同时在虚线模块内downsample不为空，其他实线模块downsample为空。
4. 一个平均池化层，stride=1；
5. 全连接层

根据上述分析，残差网络实现代码为

class ResNet(nn.Module): def __init__(self, block, layers, num_classes=1000): self.inplanes = 64 super(ResNet, self).__init__() self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False) self.bn1 = nn.BatchNorm2d(64) self.relu = nn.ReLU(inplace=True) self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=0, ceil_mode=True) # change self.layer2 = self._make_layer(block, 128, layers[1], stride=2) self.layer3 = self._make_layer(block, 256, layers[2], stride=2) self.layer4 = self._make_layer(block, 512, layers[3], stride=2) self.avgpool = nn.AvgPool2d(7) self.fc = nn.Linear(512 * block.expansion, num_classes) for m in self.modules(): if isinstance(m, nn.Conv2d): n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels m.weight.data.normal_(0, math.sqrt(2. / n)) elif isinstance(m, nn.BatchNorm2d): m.weight.data.fill_(1) m.bias.data.zero_() def _make_layer(self, block, planes, blocks, stride=1): downsample = None if stride != 1 or self.inplanes != planes * block.expansion: downsample = nn.Sequential( nn.Conv2d(self.inplanes, planes * block.expansion, kernel_size=1, stride=stride, bias=False), nn.BatchNorm2d(planes * block.expansion),) layers = [] layers.append(block(self.inplanes, planes, stride, downsample)) self.inplanes = planes * block.expansion for i in range(1, blocks): layers.append(block(self.inplanes, planes)) return nn.Sequential(*layers) def forward(self, x): x = self.conv1(x) x = self.bn1(x) x = self.relu(x) x = self.maxpool(x) x = self.layer1(x) x = self.layer2(x) x = self.layer3(x) x = self.layer4(x) x = self.avgpool(x) x = x.view(x.size(0), -1) x = self.fc(x) return x

具体的残差网络实现

建立resnet类以后，残差网络就非常容易实现了。典型的残差网络有18层、34层、50层、101层和152层几种，网络架构如下图所示

具体实现代码为

18层残差网络

def resnet18(pretrained=False): """Constructs a ResNet-18 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(BasicBlock, [2, 2, 2, 2]) if pretrained: model.load_state_dict(model_zoo.load_url(model_urls['resnet18'])) return model

34层残差网络

def resnet34(pretrained=False): """Constructs a ResNet-34 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(BasicBlock, [3, 4, 6, 3]) if pretrained: model.load_state_dict(model_zoo.load_url(model_urls['resnet34'])) return model

50层残差网络

def resnet50(pretrained=False): """Constructs a ResNet-50 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(Bottleneck, [3, 4, 6, 3]) if pretrained: model.load_state_dict(model_zoo.load_url(model_urls['resnet50'])) return model

101层残差网络

def resnet101(pretrained=False): """Constructs a ResNet-101 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(Bottleneck, [3, 4, 23, 3]) if pretrained: model.load_state_dict(model_zoo.load_url(model_urls['resnet101'])) return model

152层残差网络

def resnet152(pretrained=False): """Constructs a ResNet-152 model. Args: pretrained (bool): If True, returns a model pre-trained on ImageNet """ model = ResNet(Bottleneck, [3, 8, 36, 3]) if pretrained: model.load_state_dict(model_zoo.load_url(model_urls['resnet152'])) return model