一、前言

MobileOne论文：https://arxiv.org/abs/2206.04040
MobileOne github：https://github.com/apple/ml-mobileone

二、基本原理

使用Reparameterize重参数化实现模型的轻量化，基本模块如下图所示。

三、改进方法

说明： 该部分的改进代码尽可能地根据官方代码的写法与YOLOv7项目进行整合；

3.1 改进分析

通过阅读MobileOne源码和结合论文中Table2可以发现以下两点：
（1）Table2中Block Type全写为MobileOne Block，但在源码中的Stage1和后面的Block是稍有不同的，因此在3.2改进YOLOv7时中使用MobileOne Block和MobileOne进行区分；
（2）源码将Stage4和Stage5写在了一起，因此在换Backbone时我们也写在一起，因此在yaml中会看到Stage1后面Blocks个数为【2，8，10，1】

3.2 实现步骤

步骤一：构建MobileOneBlock、MobileOne、SEBlock、reparameterize模块
在项目文件中的models/common.py中加入以下代码

#====MobileOne====#
import copy as copy2   # 为防止与common原来引入的copy冲突, for mobileone reparameterize
from typing import Optional, List, Tupleclass SEBlock(nn.Module):""" Squeeze and Excite module.https://arxiv.org/pdf/1709.01507.pdf"""def __init__(self, in_channels: int, rd_ratio: float = 0.0625) -> None:""" Construct a Squeeze and Excite Module.:param in_channels: Number of input channels.:param rd_ratio: Input channel reduction ratio."""super(SEBlock, self).__init__()self.reduce = nn.Conv2d(in_channels=in_channels,out_channels=int(in_channels * rd_ratio), kernel_size=1, stride=1, bias=True)self.expand = nn.Conv2d(in_channels=int(in_channels * rd_ratio),out_channels=in_channels, kernel_size=1, stride=1, bias=True)def forward(self, inputs: torch.Tensor) -> torch.Tensor:""" Apply forward pass. """b, c, h, w = inputs.size()x = F.avg_pool2d(inputs, kernel_size=[h, w])x = self.reduce(x)x = F.relu(x)x = self.expand(x)x = torch.sigmoid(x)x = x.view(-1, c, 1, 1)return inputs * xclass MobileOneBlock(nn.Module):""" MobileOne building block. https://arxiv.org/pdf/2206.04040.pdf"""def __init__(self, in_channels: int, out_channels: int, kernel_size: int, stride: int = 1,padding: int = 0, dilation: int = 1, groups: int = 1, use_se: bool = False, num_conv_branches: int = 1, inference_mode: bool = False) -> None:""" Construct a MobileOneBlock module.:param in_channels: Number of channels in the input.:param out_channels: Number of channels produced by the block.:param kernel_size: Size of the convolution kernel.:param stride: Stride size.:param padding: Zero-padding size.:param dilation: Kernel dilation factor.:param groups: Group number.:param inference_mode: If True, instantiates model in inference mode.:param use_se: Whether to use SE-ReLU activations.:param num_conv_branches: Number of linear conv branches."""super(MobileOneBlock, self).__init__()self.inference_mode = inference_modeself.groups = groupsself.stride = strideself.kernel_size = kernel_sizeself.in_channels = in_channelsself.out_channels = out_channelsself.num_conv_branches = num_conv_branches  # 4# Check if SE-ReLU is requestedif use_se:self.se = SEBlock(out_channels)else:self.se = nn.Identity()self.activation = nn.ReLU()if inference_mode:self.reparam_conv = nn.Conv2d(in_channels=in_channels, out_channels=out_channels, kernel_size=kernel_size,stride=stride, padding=padding, dilation=dilation, groups=groups, bias=True)else:# Re-parameterizable skip connectionself.rbr_skip = nn.BatchNorm2d(num_features=in_channels) if out_channels == in_channels and stride == 1 else None   # BN skip# Re-parameterizable conv branchesrbr_conv = list()for _ in range(self.num_conv_branches):rbr_conv.append(self._conv_bn(kernel_size=kernel_size, padding=padding))self.rbr_conv = nn.ModuleList(rbr_conv)# Re-parameterizable scale branchself.rbr_scale = Noneif kernel_size > 1:self.rbr_scale = self._conv_bn(kernel_size=1, padding=0)def forward(self, x: torch.Tensor) -> torch.Tensor:""" Apply forward pass. """# Inference mode forward pass.if self.inference_mode:return self.activation(self.se(self.reparam_conv(x)))# Multi-branched train-time forward pass.# Skip branch outputidentity_out = 0if self.rbr_skip is not None:identity_out = self.rbr_skip(x)# Scale branch outputscale_out = 0if self.rbr_scale is not None:scale_out = self.rbr_scale(x)# Other branchesout = scale_out + identity_outfor ix in range(self.num_conv_branches):out += self.rbr_conv[ix](x)return self.activation(self.se(out))def reparameterize(self):""" Following works like `RepVGG: Making VGG-style ConvNets Great Again` -https://arxiv.org/pdf/2101.03697.pdf. We re-parameterize multi-branchedarchitecture used at training time to obtain a plain CNN-like structurefor inference."""if self.inference_mode:returnkernel, bias = self._get_kernel_bias()self.reparam_conv = nn.Conv2d(in_channels=self.rbr_conv[0].conv.in_channels,out_channels=self.rbr_conv[0].conv.out_channels,kernel_size=self.rbr_conv[0].conv.kernel_size,stride=self.rbr_conv[0].conv.stride,padding=self.rbr_conv[0].conv.padding,dilation=self.rbr_conv[0].conv.dilation,groups=self.rbr_conv[0].conv.groups,bias=True)self.reparam_conv.weight.data = kernelself.reparam_conv.bias.data = bias# Delete un-used branchesfor para in self.parameters():para.detach_()self.__delattr__('rbr_conv')self.__delattr__('rbr_scale')if hasattr(self, 'rbr_skip'):self.__delattr__('rbr_skip')self.inference_mode = Truedef _get_kernel_bias(self) -> Tuple[torch.Tensor, torch.Tensor]:""" Method to obtain re-parameterized kernel and bias.Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L83:return: Tuple of (kernel, bias) after fusing branches."""# get weights and bias of scale branchkernel_scale = 0bias_scale = 0if self.rbr_scale is not None:kernel_scale, bias_scale = self._fuse_bn_tensor(self.rbr_scale)# Pad scale branch kernel to match conv branch kernel size.pad = self.kernel_size // 2kernel_scale = torch.nn.functional.pad(kernel_scale, [pad, pad, pad, pad])# get weights and bias of skip branchkernel_identity = 0bias_identity = 0if self.rbr_skip is not None:kernel_identity, bias_identity = self._fuse_bn_tensor(self.rbr_skip)# get weights and bias of conv brancheskernel_conv = 0bias_conv = 0for ix in range(self.num_conv_branches):_kernel, _bias = self._fuse_bn_tensor(self.rbr_conv[ix])kernel_conv += _kernelbias_conv += _biaskernel_final = kernel_conv + kernel_scale + kernel_identitybias_final = bias_conv + bias_scale + bias_identityreturn kernel_final, bias_finaldef _fuse_bn_tensor(self, branch) -> Tuple[torch.Tensor, torch.Tensor]:""" Method to fuse batchnorm layer with preceeding conv layer.Reference: https://github.com/DingXiaoH/RepVGG/blob/main/repvgg.py#L95:param branch::return: Tuple of (kernel, bias) after fusing batchnorm."""if isinstance(branch, nn.Sequential):kernel = branch.conv.weightrunning_mean = branch.bn.running_meanrunning_var = branch.bn.running_vargamma = branch.bn.weightbeta = branch.bn.biaseps = branch.bn.epselse:assert isinstance(branch, nn.BatchNorm2d)if not hasattr(self, 'id_tensor'):input_dim = self.in_channels // self.groupskernel_value = torch.zeros((self.in_channels, input_dim, self.kernel_size, self.kernel_size),dtype=branch.weight.dtype, device=branch.weight.device)for i in range(self.in_channels):kernel_value[i, i % input_dim,self.kernel_size // 2, self.kernel_size // 2] = 1self.id_tensor = kernel_valuekernel = self.id_tensorrunning_mean = branch.running_meanrunning_var = branch.running_vargamma = branch.weightbeta = branch.biaseps = branch.epsstd = (running_var + eps).sqrt()t = (gamma / std).reshape(-1, 1, 1, 1)return kernel * t, beta - running_mean * gamma / stddef _conv_bn(self, kernel_size: int, padding: int) -> nn.Sequential:""" Helper method to construct conv-batchnorm layers.:param kernel_size: Size of the convolution kernel.:param padding: Zero-padding size.:return: Conv-BN module."""mod_list = nn.Sequential()mod_list.add_module('conv', nn.Conv2d(in_channels=self.in_channels,out_channels=self.out_channels,kernel_size=kernel_size, stride=self.stride, padding=padding, groups=self.groups, bias=False))mod_list.add_module('bn', nn.BatchNorm2d(num_features=self.out_channels))return mod_listclass MobileOne(nn.Module):""" MobileOne Model  https://arxiv.org/pdf/2206.04040.pdf """def __init__(self,in_channels, out_channels,num_blocks_per_stage = 2, num_conv_branches: int = 1,use_se: bool = False, num_se: int = 0,inference_mode: bool = False, ) -> None:""" Construct MobileOne model.:param num_blocks_per_stage: List of number of blocks per stage.:param num_classes: Number of classes in the dataset.:param width_multipliers: List of width multiplier for blocks in a stage.:param inference_mode: If True, instantiates model in inference mode.:param use_se: Whether to use SE-ReLU activations.:param num_conv_branches: Number of linear conv branches."""super().__init__()self.inference_mode = inference_modeself.use_se = use_seself.num_conv_branches = num_conv_branchesself.stage = self._make_stage(in_channels, out_channels, num_blocks_per_stage, num_se_blocks= num_se if use_se else 0)# planes指输出通道def _make_stage(self, in_channels, out_channels,  num_blocks: int, num_se_blocks: int) -> nn.Sequential:""" Build a stage of MobileOne model.:param planes: Number of output channels.:param num_blocks: Number of blocks in this stage.:param num_se_blocks: Number of SE blocks in this stage.:return: A stage of MobileOne model."""# Get strides for all layersstrides = [2] + [1]*(num_blocks-1)blocks = []for ix, stride in enumerate(strides):  # 用于训练几个blocksuse_se = Falseif num_se_blocks > num_blocks:raise ValueError("Number of SE blocks cannot " "exceed number of layers.")if ix >= (num_blocks - num_se_blocks):use_se = True# Depthwise convblocks.append(MobileOneBlock(in_channels=in_channels, out_channels=in_channels,kernel_size=3, stride=stride, padding=1, groups=in_channels,inference_mode=self.inference_mode, use_se=use_se, num_conv_branches=self.num_conv_branches))# Pointwise convblocks.append(MobileOneBlock(in_channels=in_channels, out_channels=out_channels,kernel_size=1, stride=1, padding=0, groups=1,inference_mode=self.inference_mode, use_se=use_se, num_conv_branches=self.num_conv_branches))in_channels = out_channelsreturn nn.Sequential(*blocks)def forward(self, x: torch.Tensor) -> torch.Tensor:""" Apply forward pass. """x = self.stage(x)return xdef reparameterize_model(model: torch.nn.Module) -> nn.Module:""" Method returns a model where a multi-branched structureused in training is re-parameterized into a single branchfor inference.:param model: MobileOne model in train mode.:return: MobileOne model in inference mode."""# Avoid editing original graphmodel = copy2.deepcopy(model)for module in model.modules():if hasattr(module, 'reparameterize'):module.reparameterize()return model

步骤二：在yolo.py的parse_model添加Mobileone的构建块

  elif m in [MobileOneBlock, MobileOne]:c1, c2 = ch[f], args[0]args = [c1, c2, *args[1:]]

步骤三：创建新的模型文件
此处以更换yolov7-tiny的backbone为例，且修改为mobileone中的ms0模型，命名yolov7-tiny-ms0.yaml

# parameters
nc: 3  # number of classes
depth_multiple: 1.0  # model depth multiple
width_multiple: 1.0  # layer channel multiple# anchors
anchors:- [10,13, 16,30, 33,23]  # P3/8- [30,61, 62,45, 59,119]  # P4/16- [116,90, 156,198, 373,326]  # P5/32# yolov7-tiny backbone
backbone:# [from, number, module, args] c2, k=1, s=1, p=None, g=1, act=True[[-1, 1, MobileOneBlock, [48, 3, 2, 1]],           # 0[-1, 1, MobileOne, [48, 2, 4, False, 0]],   # MobileOne [out_channels, num_blocks, num_conv_branches, use_se, num_se, inference_mode][-1, 1, MobileOne, [128, 8, 4, False, 0]],[-1, 1, MobileOne, [256, 10, 4, False, 0]],[ -1, 1, MobileOne, [512, 1, 4, False, 0]],  # 4]# yolov7-tiny head
head:[[-1, 1, Conv, [256, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-2, 1, Conv, [256, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, SP, [5]],[-2, 1, SP, [9]],[-3, 1, SP, [13]],[[-1, -2, -3, -4], 1, Concat, [1]],[-1, 1, Conv, [256, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -7], 1, Concat, [1]],[-1, 1, Conv, [256, 1, 1, None, 1, nn.LeakyReLU(0.1)]],  # 13[-1, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, nn.Upsample, [None, 2, 'nearest']],[3, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]], # route backbone P4[[-1, -2], 1, Concat, [1]],[-1, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-2, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [64, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [64, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -2, -3, -4], 1, Concat, [1]],[-1, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]],  # 23[-1, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, nn.Upsample, [None, 2, 'nearest']],[2, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -2], 1, Concat, [1]],                              # 27[-1, 1, Conv, [32, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-2, 1, Conv, [32, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [32, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [32, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -2, -3, -4], 1, Concat, [1]],[-1, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],  # 33[-1, 1, Conv, [128, 3, 2, None, 1, nn.LeakyReLU(0.1)]],[[-1, 23], 1, Concat, [1]],[-1, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-2, 1, Conv, [64, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [64, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [64, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -2, -3, -4], 1, Concat, [1]],[-1, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]],  # 41[-1, 1, Conv, [256, 3, 2, None, 1, nn.LeakyReLU(0.1)]],[[-1, 13], 1, Concat, [1]],[-1, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-2, 1, Conv, [128, 1, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [128, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[-1, 1, Conv, [128, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[[-1, -2, -3, -4], 1, Concat, [1]],[-1, 1, Conv, [256, 1, 1, None, 1, nn.LeakyReLU(0.1)]],  # 49[33, 1, Conv, [128, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[41, 1, Conv, [256, 3, 1, None, 1, nn.LeakyReLU(0.1)]],[49, 1, Conv, [512, 3, 1, None, 1, nn.LeakyReLU(0.1)]], # 52[[50,51,52], 1, IDetect, [nc, anchors]],   # Detect(P3, P4, P5)]

步骤五：推理部分reparameterize
在yolo.py文件中的Model类中的fuse方法，加入MobileOne和MobileOneBlock部分

    def fuse(self):  # fuse model Conv2d() + BatchNorm2d() layersprint('Fusing layers... ')for m in self.model.modules():if isinstance(m, RepConv):#print(f" fuse_repvgg_block")m.fuse_repvgg_block()elif isinstance(m, RepConv_OREPA):#print(f" switch_to_deploy")m.switch_to_deploy()#======该部分elif isinstance(m, (MobileOne, MobileOneBlock)) and hasattr(m, 'reparameterize'):m.reparameterize()#=======elif type(m) is Conv and hasattr(m, 'bn'):m.conv = fuse_conv_and_bn(m.conv, m.bn)  # update convdelattr(m, 'bn')  # remove batchnormm.forward = m.fuseforward  # update forwardelif isinstance(m, (IDetect, IAuxDetect)):m.fuse()m.forward = m.fuseforwardself.info()return self

完成以上5步就可以正常开始训练和测试了~

四、预训练权重

该部分的与训练权重是在MobileOne官方的MobileOne-ms0的官方预训练权重，已兼容YOLOv7项目。
link：https://github.com/uniquechow/YOLO_series_doc/tree/main/lightweight/MobileOne

若有其他问题，可私信交流~~~

【YOLOv7改进轻量化】第一章——引入轻量化骨干网络MobileOne相关推荐

计算机网络杨庚第一章答案,《计算机通信与网络》习题答案
杨庚等编著第一章习题解答 1.1 什么是计算机网络? 答: 我们可以把计算机网络定义为:把分布在不同地点且具有独立功能的多个计算机,通过通信设备和线路连接起来,在功能完善的网络软件运行下,以实现网 ...
计算机网络第七版总结报告,计算机网络（第七版）第一章总结（超详细！）
第一章概述 1.三大类网络电信网络有线电视网络计算机网络 2.互联网基本特点连通性远距交换信息共享资源共享 3.计算机网络由若干结点和连接这些结点的链路组成 4.网络和互联网网络把 ...
计算机网络跨考看,计算机网络试题第一章概述_跨考网
一.填空题 1计算机网络是能够相互( )的互连起来的自治计算机系统的集合. (第一章.概述.知识点:计算机网络的概念答案:资源共享) 2二十世纪七十年代( )的出现是计算机网络发展 ...
网络经济学期末复习——第一章导论
网络经济学期末复习 --第一章.导论一.课程介绍二.课本选择三.思维导图--第一章导论四.温馨提示一.课程介绍网络经济学简介: 网络经济学是近几年的一门新兴的研究网络环境下经济活动的交叉 ...
计算机网络知识点归纳第一章概述
第一章-概述互联网的概述网络的网络计算机网络(简称为"网络") 由若干节点(node)和链接这些节点的链路 (Link)组成. 网络中的节点可以是:计算机,集线器,交换机或路 ...
东北大学软件项目管理与过程改进复习提纲（2020）——第一章
文章目录第一章总论 1.1 引言 1.2 什么是项目 1.2.1 IT 项目 1.2.2 项目属性 1.2.3 项目约束 1.3 什么是项目管理 1.3.1 项目干系人 1.3.2 项目管理知识领 ...
学习《深入浅出python量化交易交易实战》第一章（笔记）
1.学习<深入浅出python量化交易交易实战>第一章记录学习过程中的代码和一些坑 1.1 基础(名词解释) 1.1.1 CAPM (Capital Asset Pricing Moda ...
帮我写一篇C语言课程的课程实施报告，第一章写整体教学设计，第二章写教学实施流程，第三章写学习效果，第4章写特色与创新，第5章写教学反思与改进...
很高兴为您写一篇 C 语言课程的课程实施报告. 第一章整体教学设计 C 语言课程的整体教学设计旨在帮助学生了解 C 语言的基础知识和编程技巧,包括语言结构.数据类型.运算符.控制结构.函数.指针.内 ...
css层叠样式表基础学习笔记--第一章 css简介及引入
第一章 css简介及引入 1-01 css简介 1-02 css优缺点 1-03 css书写格式 1-04 css引入格式行内样式内部样式外部样式导入样式 1-05 css注释 1-01 cs ...

【YOLOv7改进轻量化】第一章——引入轻量化骨干网络MobileOne