聊一聊计算机视觉中常用的注意力机制以及Pytorch代码实现

注意力机制（Attention）是深度学习中常用的tricks，可以在模型原有的基础上直接插入，进一步增强你模型的性能。注意力机制起初是作为自然语言处理中的工作Attention Is All You Need被大家所熟知，从而也引发了一系列的XX is All You Need的论文命题，SENET-Squeeze-and-Excitation Networks是注意力机制在计算机视觉中应用的早期工作之一，并获得了2017年imagenet，同时也是最后一届Imagenet比赛的冠军，后面就又出现了各种各样的注意力机制，应用在计算机视觉的任务中，今天我们就来一起聊一聊计算机视觉中常用的注意力机制以及他们对应的Pytorch代码实现，另外我还使用这些注意力机制做了一些目标检测的实验，实验效果我也一并放在博客中，大家可以一起对自己感兴趣的部分讨论讨论。

新出的手把手教程，感兴趣的兄弟们快去自己动手试试看！

手把手教你使用YOLOV5训练自己的目标检测模型-口罩检测-视频教程_dejahu的博客-CSDN博客

这里是我数据集的基本情况，这里我使用的是交通标志检测的数据集

CocoDataset Train dataset with number of images 2226, and instance counts:
+------------+-------+-----------+-------+-----------+-------+-----------------------------+-------+---------------------+-------+
| category   | count | category  | count | category  | count | category                    | count | category            | count |
+------------+-------+-----------+-------+-----------+-------+-----------------------------+-------+---------------------+-------+
| 0 [red_tl] | 1465  | 1 [arr_s] | 1133  | 2 [arr_l] | 638   | 3 [no_driving_mark_allsort] | 622   | 4 [no_parking_mark] | 1142  |
+------------+-------+-----------+-------+-----------+-------+-----------------------------+-------+---------------------+-------+

baseline选择的是fasterrcnn，实验的结果如下：

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.341Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.502Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.400Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.115Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.473Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.655Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.417Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.417Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.417Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.156Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.570Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.726

如果大家遇到论文下载比较慢

推荐使用中科院的 arxiv 镜像: http://xxx.itp.ac.cn, 国内网络能流畅访问
简单直接的方法是, 把要访问 arxiv 链接中的域名从 https://arxiv.org 换成 http://xxx.itp.ac.cn , 比如:

从 https://arxiv.org/abs/1901.07249 改为 http://xxx.itp.ac.cn/abs/1901.07249

1. SeNet: Squeeze-and-Excitation Attention

论文地址：https://arxiv.org/abs/1709.01507

网络结构

对通道做注意力机制，通过全连接层对每个通道进行加权。

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import initclass SEAttention(nn.Module):def __init__(self, channel=512, reduction=16):super().__init__()self.avg_pool = nn.AdaptiveAvgPool2d(1)self.fc = nn.Sequential(nn.Linear(channel, channel // reduction, bias=False),nn.ReLU(inplace=True),nn.Linear(channel // reduction, channel, bias=False),nn.Sigmoid())def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):b, c, _, _ = x.size()y = self.avg_pool(x).view(b, c)y = self.fc(y).view(b, c, 1, 1)return x * y.expand_as(x)if __name__ == '__main__':input = torch.randn(50, 512, 7, 7)se = SEAttention(channel=512, reduction=8)output = se(input)print(output.shape)

实验结果

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.338Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.511Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.375Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.126Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.458Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.696Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.411Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.411Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.411Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.163Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.551Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.758

2. （有用）CBAM: Convolutional Block Attention Module

论文地址：CBAM: Convolutional Block Attention Module

网络结构

对通道方向上做注意力机制之后再对空间方向上做注意力机制

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import initclass ChannelAttention(nn.Module):def __init__(self, channel, reduction=16):super().__init__()self.maxpool = nn.AdaptiveMaxPool2d(1)self.avgpool = nn.AdaptiveAvgPool2d(1)self.se = nn.Sequential(nn.Conv2d(channel, channel // reduction, 1, bias=False),nn.ReLU(),nn.Conv2d(channel // reduction, channel, 1, bias=False))self.sigmoid = nn.Sigmoid()def forward(self, x):max_result = self.maxpool(x)avg_result = self.avgpool(x)max_out = self.se(max_result)avg_out = self.se(avg_result)output = self.sigmoid(max_out + avg_out)return outputclass SpatialAttention(nn.Module):def __init__(self, kernel_size=7):super().__init__()self.conv = nn.Conv2d(2, 1, kernel_size=kernel_size, padding=kernel_size // 2)self.sigmoid = nn.Sigmoid()def forward(self, x):max_result, _ = torch.max(x, dim=1, keepdim=True)avg_result = torch.mean(x, dim=1, keepdim=True)result = torch.cat([max_result, avg_result], 1)output = self.conv(result)output = self.sigmoid(output)return outputclass CBAMBlock(nn.Module):def __init__(self, channel=512, reduction=16, kernel_size=49):super().__init__()self.ca = ChannelAttention(channel=channel, reduction=reduction)self.sa = SpatialAttention(kernel_size=kernel_size)def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):b, c, _, _ = x.size()residual = xout = x * self.ca(x)out = out * self.sa(out)return out + residualif __name__ == '__main__':input = torch.randn(50, 512, 7, 7)kernel_size = input.shape[2]cbam = CBAMBlock(channel=512, reduction=16, kernel_size=kernel_size)output = cbam(input)print(output.shape)

实验结果

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.364Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.544Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.425Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.137Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.499Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.674Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.439Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.439Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.439Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.185Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.590Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.755

3. BAM: Bottleneck Attention Module

论文地址：https://arxiv.org/pdf/1807.06514.pdf

网络结构

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import initclass Flatten(nn.Module):def forward(self, x):return x.view(x.shape[0], -1)class ChannelAttention(nn.Module):def __init__(self, channel, reduction=16, num_layers=3):super().__init__()self.avgpool = nn.AdaptiveAvgPool2d(1)gate_channels = [channel]gate_channels += [channel // reduction] * num_layersgate_channels += [channel]self.ca = nn.Sequential()self.ca.add_module('flatten', Flatten())for i in range(len(gate_channels) - 2):self.ca.add_module('fc%d' % i, nn.Linear(gate_channels[i], gate_channels[i + 1]))self.ca.add_module('bn%d' % i, nn.BatchNorm1d(gate_channels[i + 1]))self.ca.add_module('relu%d' % i, nn.ReLU())self.ca.add_module('last_fc', nn.Linear(gate_channels[-2], gate_channels[-1]))def forward(self, x):res = self.avgpool(x)res = self.ca(res)res = res.unsqueeze(-1).unsqueeze(-1).expand_as(x)return resclass SpatialAttention(nn.Module):def __init__(self, channel, reduction=16, num_layers=3, dia_val=2):super().__init__()self.sa = nn.Sequential()self.sa.add_module('conv_reduce1',nn.Conv2d(kernel_size=1, in_channels=channel, out_channels=channel // reduction))self.sa.add_module('bn_reduce1', nn.BatchNorm2d(channel // reduction))self.sa.add_module('relu_reduce1', nn.ReLU())for i in range(num_layers):self.sa.add_module('conv_%d' % i, nn.Conv2d(kernel_size=3, in_channels=channel // reduction,out_channels=channel // reduction, padding=1, dilation=dia_val))self.sa.add_module('bn_%d' % i, nn.BatchNorm2d(channel // reduction))self.sa.add_module('relu_%d' % i, nn.ReLU())self.sa.add_module('last_conv', nn.Conv2d(channel // reduction, 1, kernel_size=1))def forward(self, x):res = self.sa(x)res = res.expand_as(x)return resclass BAMBlock(nn.Module):def __init__(self, channel=512, reduction=16, dia_val=2):super().__init__()self.ca = ChannelAttention(channel=channel, reduction=reduction)self.sa = SpatialAttention(channel=channel, reduction=reduction, dia_val=dia_val)self.sigmoid = nn.Sigmoid()def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):b, c, _, _ = x.size()sa_out = self.sa(x)ca_out = self.ca(x)weight = self.sigmoid(sa_out + ca_out)out = (1 + weight) * xreturn outif __name__ == '__main__':input = torch.randn(50, 512, 7, 7)bam = BAMBlock(channel=512, reduction=16, dia_val=2)output = bam(input)print(output.shape)

实验结果
```
无
```

4. （有用）ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

论文地址：https://arxiv.org/pdf/1910.03151.pdf

网络结构

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import init
from collections import OrderedDictclass ECAAttention(nn.Module):def __init__(self, kernel_size=3):super().__init__()self.gap = nn.AdaptiveAvgPool2d(1)self.conv = nn.Conv1d(1, 1, kernel_size=kernel_size, padding=(kernel_size - 1) // 2)self.sigmoid = nn.Sigmoid()def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):y = self.gap(x)  # bs,c,1,1y = y.squeeze(-1).permute(0, 2, 1)  # bs,1,cy = self.conv(y)  # bs,1,cy = self.sigmoid(y)  # bs,1,cy = y.permute(0, 2, 1).unsqueeze(-1)  # bs,c,1,1return x * y.expand_as(x)if __name__ == '__main__':input = torch.randn(50, 512, 7, 7)eca = ECAAttention(kernel_size=3)output = eca(input)print(output.shape)

实验结果

2021-12-17 12:18:08,911 - mmdet - INFO - Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.360Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.545Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.414Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.141Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.489Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.676Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.432Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.432Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.432Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.184Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.576Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.748

5. SA-NET: SHUFFLE ATTENTION FOR DEEP CONVOLUTIONAL NEURAL NETWORKS

论文地址：https://arxiv.org/pdf/2102.00240.pdf

网络结构

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import init
from torch.nn.parameter import Parameterclass ShuffleAttention(nn.Module):def __init__(self, channel=512, reduction=16, G=8):super().__init__()self.G = Gself.channel = channelself.avg_pool = nn.AdaptiveAvgPool2d(1)self.gn = nn.GroupNorm(channel // (2 * G), channel // (2 * G))self.cweight = Parameter(torch.zeros(1, channel // (2 * G), 1, 1))self.cbias = Parameter(torch.ones(1, channel // (2 * G), 1, 1))self.sweight = Parameter(torch.zeros(1, channel // (2 * G), 1, 1))self.sbias = Parameter(torch.ones(1, channel // (2 * G), 1, 1))self.sigmoid = nn.Sigmoid()def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)@staticmethoddef channel_shuffle(x, groups):b, c, h, w = x.shapex = x.reshape(b, groups, -1, h, w)x = x.permute(0, 2, 1, 3, 4)# flattenx = x.reshape(b, -1, h, w)return xdef forward(self, x):b, c, h, w = x.size()# group into subfeaturesx = x.view(b * self.G, -1, h, w)  # bs*G,c//G,h,w# channel_splitx_0, x_1 = x.chunk(2, dim=1)  # bs*G,c//(2*G),h,w# channel attentionx_channel = self.avg_pool(x_0)  # bs*G,c//(2*G),1,1x_channel = self.cweight * x_channel + self.cbias  # bs*G,c//(2*G),1,1x_channel = x_0 * self.sigmoid(x_channel)# spatial attentionx_spatial = self.gn(x_1)  # bs*G,c//(2*G),h,wx_spatial = self.sweight * x_spatial + self.sbias  # bs*G,c//(2*G),h,wx_spatial = x_1 * self.sigmoid(x_spatial)  # bs*G,c//(2*G),h,w# concatenate along channel axisout = torch.cat([x_channel, x_spatial], dim=1)  # bs*G,c//G,h,wout = out.contiguous().view(b, -1, h, w)# channel shuffleout = self.channel_shuffle(out, 2)return outif __name__ == '__main__':input = torch.randn(50, 512, 7, 7)se = ShuffleAttention(channel=512, G=8)output = se(input)print(output.shape)

实验结果

 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.350Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.523Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.401Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.123Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.479Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.662Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.160Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.576Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.733

6. Polarized Self-Attention: Towards High-quality Pixel-wise Regression

论文地址：https://arxiv.org/abs/2107.00782

网络结构

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import initclass ParallelPolarizedSelfAttention(nn.Module):def __init__(self, channel=512):super().__init__()self.ch_wv = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.ch_wq = nn.Conv2d(channel, 1, kernel_size=(1, 1))self.softmax_channel = nn.Softmax(1)self.softmax_spatial = nn.Softmax(-1)self.ch_wz = nn.Conv2d(channel // 2, channel, kernel_size=(1, 1))self.ln = nn.LayerNorm(channel)self.sigmoid = nn.Sigmoid()self.sp_wv = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.sp_wq = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.agp = nn.AdaptiveAvgPool2d((1, 1))def forward(self, x):b, c, h, w = x.size()# Channel-only Self-Attentionchannel_wv = self.ch_wv(x)  # bs,c//2,h,wchannel_wq = self.ch_wq(x)  # bs,1,h,wchannel_wv = channel_wv.reshape(b, c // 2, -1)  # bs,c//2,h*wchannel_wq = channel_wq.reshape(b, -1, 1)  # bs,h*w,1channel_wq = self.softmax_channel(channel_wq)channel_wz = torch.matmul(channel_wv, channel_wq).unsqueeze(-1)  # bs,c//2,1,1channel_weight = self.sigmoid(self.ln(self.ch_wz(channel_wz).reshape(b, c, 1).permute(0, 2, 1))).permute(0, 2,1).reshape(b, c, 1, 1)  # bs,c,1,1channel_out = channel_weight * x# Spatial-only Self-Attentionspatial_wv = self.sp_wv(x)  # bs,c//2,h,wspatial_wq = self.sp_wq(x)  # bs,c//2,h,wspatial_wq = self.agp(spatial_wq)  # bs,c//2,1,1spatial_wv = spatial_wv.reshape(b, c // 2, -1)  # bs,c//2,h*wspatial_wq = spatial_wq.permute(0, 2, 3, 1).reshape(b, 1, c // 2)  # bs,1,c//2spatial_wq = self.softmax_spatial(spatial_wq)spatial_wz = torch.matmul(spatial_wq, spatial_wv)  # bs,1,h*wspatial_weight = self.sigmoid(spatial_wz.reshape(b, 1, h, w))  # bs,1,h,wspatial_out = spatial_weight * xout = spatial_out + channel_outreturn outclass SequentialPolarizedSelfAttention(nn.Module):def __init__(self, channel=512):super().__init__()self.ch_wv = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.ch_wq = nn.Conv2d(channel, 1, kernel_size=(1, 1))self.softmax_channel = nn.Softmax(1)self.softmax_spatial = nn.Softmax(-1)self.ch_wz = nn.Conv2d(channel // 2, channel, kernel_size=(1, 1))self.ln = nn.LayerNorm(channel)self.sigmoid = nn.Sigmoid()self.sp_wv = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.sp_wq = nn.Conv2d(channel, channel // 2, kernel_size=(1, 1))self.agp = nn.AdaptiveAvgPool2d((1, 1))def forward(self, x):b, c, h, w = x.size()# Channel-only Self-Attentionchannel_wv = self.ch_wv(x)  # bs,c//2,h,wchannel_wq = self.ch_wq(x)  # bs,1,h,wchannel_wv = channel_wv.reshape(b, c // 2, -1)  # bs,c//2,h*wchannel_wq = channel_wq.reshape(b, -1, 1)  # bs,h*w,1channel_wq = self.softmax_channel(channel_wq)channel_wz = torch.matmul(channel_wv, channel_wq).unsqueeze(-1)  # bs,c//2,1,1channel_weight = self.sigmoid(self.ln(self.ch_wz(channel_wz).reshape(b, c, 1).permute(0, 2, 1))).permute(0, 2,1).reshape(b, c, 1, 1)  # bs,c,1,1channel_out = channel_weight * x# Spatial-only Self-Attentionspatial_wv = self.sp_wv(channel_out)  # bs,c//2,h,wspatial_wq = self.sp_wq(channel_out)  # bs,c//2,h,wspatial_wq = self.agp(spatial_wq)  # bs,c//2,1,1spatial_wv = spatial_wv.reshape(b, c // 2, -1)  # bs,c//2,h*wspatial_wq = spatial_wq.permute(0, 2, 3, 1).reshape(b, 1, c // 2)  # bs,1,c//2spatial_wq = self.softmax_spatial(spatial_wq)spatial_wz = torch.matmul(spatial_wq, spatial_wv)  # bs,1,h*wspatial_weight = self.sigmoid(spatial_wz.reshape(b, 1, h, w))  # bs,1,h,wspatial_out = spatial_weight * channel_outreturn spatial_outif __name__ == '__main__':input = torch.randn(1, 512, 7, 7)psa = SequentialPolarizedSelfAttention(channel=512)output = psa(input)print(output.shape)

实验结果

2021-12-16 20:30:36,981 - mmdet - INFO - Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.346Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.522Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.385Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.123Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.474Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.676Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.422Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.422Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.422Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.170Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.570Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.743

7. Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks

论文地址：https://arxiv.org/pdf/1905.09646.pdf

网络结构

主要是用在语义分割上，所以在检测上的效果一般，没有带来多少提升

Pytorch代码

import numpy as np
import torch
from torch import nn
from torch.nn import initclass SpatialGroupEnhance(nn.Module):def __init__(self, groups):super().__init__()self.groups = groupsself.avg_pool = nn.AdaptiveAvgPool2d(1)self.weight = nn.Parameter(torch.zeros(1, groups, 1, 1))self.bias = nn.Parameter(torch.zeros(1, groups, 1, 1))self.sig = nn.Sigmoid()self.init_weights()def init_weights(self):for m in self.modules():if isinstance(m, nn.Conv2d):init.kaiming_normal_(m.weight, mode='fan_out')if m.bias is not None:init.constant_(m.bias, 0)elif isinstance(m, nn.BatchNorm2d):init.constant_(m.weight, 1)init.constant_(m.bias, 0)elif isinstance(m, nn.Linear):init.normal_(m.weight, std=0.001)if m.bias is not None:init.constant_(m.bias, 0)def forward(self, x):b, c, h, w = x.shapex = x.view(b * self.groups, -1, h, w)  # bs*g,dim//g,h,wxn = x * self.avg_pool(x)  # bs*g,dim//g,h,wxn = xn.sum(dim=1, keepdim=True)  # bs*g,1,h,wt = xn.view(b * self.groups, -1)  # bs*g,h*wt = t - t.mean(dim=1, keepdim=True)  # bs*g,h*wstd = t.std(dim=1, keepdim=True) + 1e-5t = t / std  # bs*g,h*wt = t.view(b, self.groups, h, w)  # bs,g,h*wt = t * self.weight + self.bias  # bs,g,h*wt = t.view(b * self.groups, 1, h, w)  # bs*g,1,h*wx = x * self.sig(t)x = x.view(b, c, h, w)return xif __name__ == '__main__':input = torch.randn(50, 512, 7, 7)sge = SpatialGroupEnhance(groups=8)output = sge(input)print(output.shape)

实验结果

2021-12-16 21:39:42,785 - mmdet - INFO - Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.342Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.516Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.381Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.117Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.474Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.652Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.415Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.415Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.415Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.155Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.565Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.718

8. Coordinate Attention for Efficient Mobile Network Design

论文地址：https://arxiv.org/abs/2103.02907

网络结构

主要应用在轻量级网络上，在resnet系列上效果不好。

Pytorch代码

import torch
import torch.nn as nn
import torch.nn.functional as Fclass h_sigmoid(nn.Module):def __init__(self, inplace=True):super(h_sigmoid, self).__init__()self.relu = nn.ReLU6(inplace=inplace)def forward(self, x):return self.relu(x + 3) / 6class h_swish(nn.Module):def __init__(self, inplace=True):super(h_swish, self).__init__()self.sigmoid = h_sigmoid(inplace=inplace)def forward(self, x):return x * self.sigmoid(x)class CoordAtt(nn.Module):def __init__(self, inp, oup, reduction=32):super(CoordAtt, self).__init__()self.pool_h = nn.AdaptiveAvgPool2d((None, 1))self.pool_w = nn.AdaptiveAvgPool2d((1, None))mip = max(8, inp // reduction)self.conv1 = nn.Conv2d(inp, mip, kernel_size=1, stride=1, padding=0)self.bn1 = nn.BatchNorm2d(mip)self.act = h_swish()self.conv_h = nn.Conv2d(mip, oup, kernel_size=1, stride=1, padding=0)self.conv_w = nn.Conv2d(mip, oup, kernel_size=1, stride=1, padding=0)def forward(self, x):identity = xn, c, h, w = x.size()x_h = self.pool_h(x)x_w = self.pool_w(x).permute(0, 1, 3, 2)y = torch.cat([x_h, x_w], dim=2)y = self.conv1(y)y = self.bn1(y)y = self.act(y)x_h, x_w = torch.split(y, [h, w], dim=2)x_w = x_w.permute(0, 1, 3, 2)a_h = self.conv_h(x_h).sigmoid()a_w = self.conv_w(x_w).sigmoid()out = identity * a_w * a_hreturn out

实验结果

2021-12-16 19:04:16,776 - mmdet - INFO - Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.340Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.516Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.386Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.127Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.457Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.632Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.408Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.408Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.162Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.546Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.716

9. Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

论文地址： https://arxiv.org/abs/2112.05561

网络结构

计算量特别大，效果一般

Pytorch代码

class GAM_Attention(nn.Module):def __init__(self, in_channels, out_channels, rate=4):super(GAM_Attention, self).__init__()self.channel_attention = nn.Sequential(nn.Linear(in_channels, int(in_channels / rate)),nn.ReLU(inplace=True),nn.Linear(int(in_channels / rate), in_channels))self.spatial_attention = nn.Sequential(nn.Conv2d(in_channels, int(in_channels / rate), kernel_size=7, padding=3),nn.BatchNorm2d(int(in_channels / rate)),nn.ReLU(inplace=True),nn.Conv2d(int(in_channels / rate), out_channels, kernel_size=7, padding=3),nn.BatchNorm2d(out_channels))def forward(self, x):# print(x)b, c, h, w = x.shapex_permute = x.permute(0, 2, 3, 1).view(b, -1, c)x_att_permute = self.channel_attention(x_permute).view(b, h, w, c)x_channel_att = x_att_permute.permute(0, 3, 1, 2)x = x * x_channel_attx_spatial_att = self.spatial_attention(x).sigmoid()out = x * x_spatial_att# print(out)return out

实验结果

2021-12-16 16:14:20,693 - mmdet - INFO - Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.350Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=1000 ] = 0.530Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=1000 ] = 0.399Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.131Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.481Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.683Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=300 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=1000 ] = 0.424Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=1000 ] = 0.171Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=1000 ] = 0.575Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=1000 ] = 0.745

找到我

你可以通过这些方式来寻找我。

B站：肆十二-

CSDN：肆十二

知乎：肆十二

微博：肆十二-

现在关注以后就是老朋友喽！

聊一聊计算机视觉中常用的注意力机制附Pytorch代码实现相关推荐

【ICML2021】计算机视觉中的自注意力机制教程，谷歌伯克利出品
本教程将介绍自注意力机制在计算机视觉中的应用.Self-Attention在NLP中被广泛采用,完全注意的Transformer模型已经在很大程度上取代了RNN,现在被用于最先进的语言理解模型,如GP ...
【理论+实践】史上最全-论文中常用的图像分割评价指标-附完整代码
图像分割的评价指标非常多,论文中常用的包括像素准确率(Pixel Accuracy, PA).交并比(Intersection-Over-Union,IOU).Dice系数(Dice Coeffcie ...
神经网络 || 注意力机制的Pytorch代码实现
文章目录 1 注意力机制的诞生 2 介绍SeNet模型及Pytorch代码实现 1 注意力机制的诞生注意力机制,起初是作为自然语言处理中的工作为大家熟知,文章Attention is all you ...
【深度学习】(8) CNN中的通道注意力机制（SEnet、ECAnet），附Tensorflow完整代码
各位同学好,今天和大家分享一下attention注意力机制在CNN卷积神经网络中的应用,重点介绍三种注意力机制,及其代码复现. 在我之前的神经网络专栏的文章中也使用到过注意力机制,比如在MobileN ...
收藏 | 理解卷积神经网络中的自注意力机制
点上方蓝字计算机视觉联盟获取更多干货在右上方 ··· 设为星标 ★,与你不见不散仅作学术分享,不代表本公众号立场,侵权联系删除转载于:作者丨Shuchen Du.ronghuaiyang(译) ...
理解卷积神经网络中的自注意力机制
作者 | Shuchen Du 编译 | ronghuaiyang 转自 | AI公园导读计算机视觉中的编解码结构的局限性以及提升方法. 卷积神经网络(CNN)广泛应用于深度学习和计算机视觉算法中 ...
计算机视觉专业术语,计算机视觉中常用的术语.doc
计算机视觉中常用的术语计算机视觉常用术语中英文对照(1) 2011-06-08 21:26 ??Artificial Intelligence 认知科学和神经科学Cognitive Science ...
Java基础-Java中常用的锁机制与使用
Java基础-Java中常用的锁机制与使用锁lock或互斥mutex是一种同步机制,主要用于在存在多线程的环境中强制对资源进行访问限制.锁的主要作用为强制实施互斥排他以及并发控制策略.锁一般需要硬件 ...
Attention 扫盲：注意力机制及其 PyTorch 应用实现
点击上方"MLNLP",选择"星标"公众号重磅干货,第一时间送达来自 | 知乎作者 | Lucas 地址 | https://zhuanlan.zhihu ...

聊一聊计算机视觉中常用的注意力机制附Pytorch代码实现

聊一聊计算机视觉中常用的注意力机制以及Pytorch代码实现

1. SeNet: Squeeze-and-Excitation Attention

2. （有用）CBAM: Convolutional Block Attention Module

3. BAM: Bottleneck Attention Module

4. （有用）ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

5. SA-NET: SHUFFLE ATTENTION FOR DEEP CONVOLUTIONAL NEURAL NETWORKS

6. Polarized Self-Attention: Towards High-quality Pixel-wise Regression

7. Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks

8. Coordinate Attention for Efficient Mobile Network Design

9. Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

更多注意力

双路注意力机制-DANET

位置注意力-CCNET

找到我

聊一聊计算机视觉中常用的注意力机制附Pytorch代码实现相关推荐

最新文章

热门文章

聊一聊计算机视觉中常用的注意力机制 附Pytorch代码实现

聊一聊计算机视觉中常用的注意力机制以及Pytorch代码实现

1. SeNet: Squeeze-and-Excitation Attention

2. （有用）CBAM: Convolutional Block Attention Module

3. BAM: Bottleneck Attention Module

4. （有用）ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks

5. SA-NET: SHUFFLE ATTENTION FOR DEEP CONVOLUTIONAL NEURAL NETWORKS

6. Polarized Self-Attention: Towards High-quality Pixel-wise Regression

7. Spatial Group-wise Enhance: Improving Semantic Feature Learning in Convolutional Networks

8. Coordinate Attention for Efficient Mobile Network Design

9. Global Attention Mechanism: Retain Information to Enhance Channel-Spatial Interactions

更多注意力

双路注意力机制-DANET

位置注意力-CCNET

找到我

聊一聊计算机视觉中常用的注意力机制 附Pytorch代码实现相关推荐

最新文章

热门文章

聊一聊计算机视觉中常用的注意力机制附Pytorch代码实现

聊一聊计算机视觉中常用的注意力机制附Pytorch代码实现相关推荐