深度学习论文: An Energy and GPU-Computation Efficient Backbone Network for Object Detection及其PyTorch

An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection
PDF:https://arxiv.org/pdf/1904.09730.pdf
PyTorch: https://github.com/shanglianlm0525/PyTorch-Networks

相关文章链接：
VoVNet 深度学习论文: An Energy and GPU-Computation Efficient Backbone Network for Object Detection及其PyTorch
VoVNetV2 深度学习论文: CenterMask : Real-Time Anchor-Free Instance Segmentation及其PyTorch实现

1 概述

DenseNet用更少的参数与Flops而性能比ResNet更好，主要是因为concat比add能保留更多的信息。但是，实际上DenseNet却比ResNet要慢且消耗更多资源。

本文提出了One-short Aggregation，其实就是改变了shortcut的连接方式，从原来的前层feature与后面每一层都连接，变成了前层feature只与最后一层的连接。大大减少了连接数量。作者声称同样的结构，性能差不多，但是比DenseNet快2倍，energy consumption也减少了1.6~4.1倍。

2 Factors of Efficient Network Design(高效网络设计要素)

2-1 Memory Access Cost (MAC)

对于CNN，能耗在内存访问而不是计算上。
影响MAC的主要是是内存占用（intermediate activation memory footprint），它主要受卷积核和feature map大小的影响。

每个卷积层的MAC计算如下，

其中， h, w分别为高和宽，k为核尺寸, cic_{i}ci, coc_{o}co 分别为输入和输出通道数。由上式有

其中 B=k2∗h∗w∗ci∗coB = k^{2}*h*w*c_{i}* c_{o}B=k2∗h∗w∗ci∗co
明显可以看出当输入和输出通道数相等时，MAC开销最小。

2-2 GPU-Computation Efficiency

GPU特性是擅长parallel computation，tensor越大，GPU使用效率越高。
把大的卷积操作拆分成碎片的小操作将不利于GPU计算。
设计layer数量少的网络是更好的选择。
1x1卷积来减少计算量，不过这不利于GPU计算。

3 Network

3-1 One-Shot Aggregation

提出OSA（one-shot-aggregation）模块。在OSA module中，每一层产生两种连接，一种是通过conv和下一层连接，产生receptive field 更大的feature map，另一种是和最后的输出层相连，以聚合足够好的特征。
通过使用OSA module，5层43channels的DenseNet-40的MAC可以被减少30%（3.7M -> 2.5M）

PyTorch代码：

class OSA_module(nn.Module):def __init__(self, in_channels,mid_channels, out_channels, block_nums=5):super(OSA_module, self).__init__()self._layers = nn.ModuleList()self._layers.append(Conv3x3BNReLU(in_channels=in_channels, out_channels=mid_channels, stride=1))for idx in range(block_nums-1):self._layers.append(Conv3x3BNReLU(in_channels=mid_channels, out_channels=mid_channels, stride=1))self.conv1x1 = Conv1x1BNReLU(in_channels+mid_channels*block_nums,out_channels)def forward(self, x):outputs = []outputs.append(x)for _layer in self._layers:x = _layer(x)outputs.append(x)out = torch.cat(outputs, dim=1)out = self.conv1x1(out)return out

3-2 VoVNet

提出的各种VoVNet结构

PyTorch代码：

# !/usr/bin/env python
# -- coding: utf-8 --
# @Time : 2020/6/1 14:40
# @Author : liumin
# @File : VoVNet.pyimport torch
import torch.nn as nn
import torchvision__all__ = ['VoVNet', 'vovnet27_slim', 'vovnet39', 'vovnet57']from PIL.Image import Imagedef Conv3x3BNReLU(in_channels,out_channels,stride,groups=1):return nn.Sequential(nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, stride=stride, padding=1,groups=groups, bias=False),nn.BatchNorm2d(out_channels),nn.ReLU6(inplace=True))def Conv3x3BN(in_channels,out_channels,stride,groups):return nn.Sequential(nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, stride=stride, padding=1,groups=groups, bias=False),nn.BatchNorm2d(out_channels))def Conv1x1BNReLU(in_channels,out_channels):return nn.Sequential(nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=1, bias=False),nn.BatchNorm2d(out_channels),nn.ReLU6(inplace=True))def Conv1x1BN(in_channels,out_channels):return nn.Sequential(nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=1, stride=1, bias=False),nn.BatchNorm2d(out_channels))class OSA_module(nn.Module):def __init__(self, in_channels,mid_channels, out_channels, block_nums=5):super(OSA_module, self).__init__()self._layers = nn.ModuleList()self._layers.append(Conv3x3BNReLU(in_channels=in_channels, out_channels=mid_channels, stride=1))for idx in range(block_nums-1):self._layers.append(Conv3x3BNReLU(in_channels=mid_channels, out_channels=mid_channels, stride=1))self.conv1x1 = Conv1x1BNReLU(in_channels+mid_channels*block_nums,out_channels)def forward(self, x):outputs = []outputs.append(x)for _layer in self._layers:x = _layer(x)outputs.append(x)out = torch.cat(outputs, dim=1)out = self.conv1x1(out)return outclass VoVNet(nn.Module):def __init__(self, planes, layers, num_classes=2):super(VoVNet, self).__init__()self.groups = 1self.stage1 = nn.Sequential(Conv3x3BNReLU(in_channels=3, out_channels=64, stride=2, groups=self.groups),Conv3x3BNReLU(in_channels=64, out_channels=64, stride=1, groups=self.groups),Conv3x3BNReLU(in_channels=64, out_channels=128, stride=1, groups=self.groups),)self.stage2 = self._make_layer(planes[0][0],planes[0][1],planes[0][2],layers[0])self.stage3 = self._make_layer(planes[1][0],planes[1][1],planes[1][2],layers[1])self.stage4 = self._make_layer(planes[2][0],planes[2][1],planes[2][2],layers[2])self.stage5 = self._make_layer(planes[3][0],planes[3][1],planes[3][2],layers[3])self.avgpool = nn.AdaptiveAvgPool2d(output_size=1)self.flatten = nn.Flatten()self.dropout = nn.Dropout(p=0.2)self.linear = nn.Linear(in_features=planes[3][2], out_features=num_classes)def _make_layer(self, in_channels, mid_channels,out_channels, block_num):layers = []layers.append(nn.MaxPool2d(kernel_size=3, stride=2, padding=1))for idx in range(block_num):layers.append(OSA_module(in_channels=in_channels, mid_channels=mid_channels, out_channels=out_channels))in_channels = out_channelsreturn nn.Sequential(*layers)def init_params(self):for m in self.modules():if isinstance(m, nn.Conv2d):nn.init.kaiming_normal_(m.weight)if m.bias is not None:nn.init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.Linear):nn.init.constant_(m.weight, 1)nn.init.constant_(m.bias, 0)def forward(self, x):x = self.stage1(x)x = self.stage2(x)x = self.stage3(x)x = self.stage4(x)x = self.stage5(x)x = self.avgpool(x)x = self.flatten(x)x = self.dropout(x)out = self.linear(x)return outdef vovnet27_slim(**kwargs):planes = [[128, 64, 128],[128, 80, 256],[256, 96, 384],[384, 112, 512]]layers = [1, 1, 1, 1]model = VoVNet(planes, layers)return modeldef vovnet39(**kwargs):planes = [[128, 128, 256],[256, 160, 512],[512, 192, 768],[768, 224, 1024]]layers = [1, 1, 2, 2]model = VoVNet(planes, layers)return modeldef vovnet57(**kwargs):planes = [[128, 128, 256],[256, 160, 512],[512, 192, 768],[768, 224, 1024]]layers = [1, 1, 4, 3]model = VoVNet(planes, layers)return modelif __name__=='__main__':model = vovnet27_slim()print(model)input = torch.randn(1, 3, 64, 64)out = model(input)print(out.shape)