引入

边缘检测系列第 5 弹，本次继续介绍经典的边缘检测模型
Crisp Edge Detection（CED）模型是前面介绍过的 HED 模型的另一种改进模型
CED 模型利用自上而下的反向细化路径，并逐渐增加特征图的分辨率以生成清晰的边缘
因为官方没有提供预训练模型，所有本次仅简单介绍一下模型的结构和代码实现

参考资料

论文：
- Deep Crisp Boundaries: From Boundaries to Higher-level Tasks
- Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
参考代码：Wangyupei/CED

引用：

@article{2019,title={Deep Crisp Boundaries: From Boundaries to Higher-Level Tasks},volume={28},ISSN={1941-0042},url={http://dx.doi.org/10.1109/TIP.2018.2874279},DOI={10.1109/tip.2018.2874279},number={3},journal={IEEE Transactions on Image Processing},publisher={Institute of Electrical and Electronics Engineers (IEEE)},author={Wang, Yupei and Zhao, Xin and Li, Yin and Huang, Kaiqi},year={2019},month={Mar},pages={1285–1298}
}

效果参考

论文效果图：

模型架构

CED 模型总体基于 HED 模型改造而来，其中做了如下几个改进：
- 将模型中的上采样操作从转置卷积插值更换为 PixelShuffle
- 添加了反向细化路径，即一个反向的从高层级特征逐步往低层级特征的边缘细化路径
- 没有多层级输出，最终的输出为融合了各层级的特征的边缘检测结果
架构图如下

PixelShuffle

正常情况下，卷积操作会使特征图的高和宽缩小。但当我们的步长即 stride = 1/r < 1 时，可以让卷积后的特征图的高和宽扩大——即分辨率增大或者叫上采样，这个新的操作叫做 PixelShuffle
更多算法细节可以参考论文：Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
原理图如下：

代码实现

骨干网络

依旧是 VGG16 这个非常经典的骨干网络

import paddle
import paddle.nn as nnfrom paddle.utils.download import get_weights_path_from_url
from typing import List__all__ = []model_urls = {'vgg16': ('https://paddle-hapi.bj.bcebos.com/models/vgg16.pdparams','89bbffc0f87d260be9b8cdc169c991c4')
}class VGG(nn.Layer):"""VGG model from`"Very Deep Convolutional Networks For Large-Scale Image Recognition" <https://arxiv.org/pdf/1409.1556.pdf>`_Args:features (nn.Layer): Vgg features create by function make_layers.num_classes (int): Output dim of last fc layer. If num_classes <=0, last fc layer will not be defined. Default: 1000.with_pool (bool): Use pool before the last three fc layer or not. Default: True.Examples:.. code-block:: pythonfrom paddle.vision.models import VGGfrom paddle.vision.models.vgg import make_layersvgg11_cfg = [64, 'M', 128, 'M', 256, 256, 'M', 512, 512, 'M', 512, 512, 'M']features = make_layers(vgg11_cfg)vgg11 = VGG(features)"""def __init__(self, features):super(VGG, self).__init__()self.features = featuresself.feat_channels = [layer._out_channelsfor layer in featuresif isinstance(layer, nn.Conv2D)]def forward(self, x):outputs = []for layer in self.features:x = layer(x)if isinstance(layer, nn.ReLU):outputs.append(x)return outputsdef make_layers(cfg, batch_norm=False):layers = []in_channels = 3for v in cfg:if v == 'M':layers += [nn.MaxPool2D(kernel_size=2, stride=2)]else:conv2d = nn.Conv2D(in_channels, v, kernel_size=3, padding=1)if batch_norm:layers += [conv2d, nn.BatchNorm2D(v), nn.ReLU()]else:layers += [conv2d, nn.ReLU()]in_channels = vreturn nn.Sequential(*layers)cfgs = {'D': [                   # return_idx64, 64,              # 0, 1'M', 128, 128,       # 2, 3'M', 256, 256, 256,  # 4, 5, 6'M', 512, 512, 512,  # 7, 8, 9'M', 512, 512, 512   # 10, 11, 12]
}def _vgg(arch, cfg, batch_norm, pretrained, **kwargs):model = VGG(make_layers(cfgs[cfg], batch_norm=batch_norm), **kwargs)if pretrained:assert arch in model_urls, "{} model do not have a pretrained model now, you should set pretrained=False".format(arch)weight_path = get_weights_path_from_url(model_urls[arch][0],model_urls[arch][1])param = paddle.load(weight_path)model.load_dict(param)return modeldef vgg16(pretrained=False, batch_norm=False, **kwargs):"""VGG 16-layer model Args:pretrained (bool): If True, returns a model pre-trained on ImageNet. Default: False.batch_norm (bool): If True, returns a model with batch_norm layer. Default: False.Examples:.. code-block:: pythonfrom paddle.vision.models import vgg16# build modelmodel = vgg16()# build vgg16 model with batch_normmodel = vgg16(batch_norm=True)"""model_name = 'vgg16'if batch_norm:model_name += ('_bn')return _vgg(model_name, 'D', batch_norm, pretrained, **kwargs)

CED 模块

一个简单的多层卷积网络，其中加入了一个 PixelShuffle 进行上采样操作

class CEDBlock(nn.Layer):def __init__(self,in_channels_up: int,in_channels_down: int,out_channels_sub: int,out_channels: int) -> None:super().__init__()self.conv_up = nn.Conv2D(in_channels=in_channels_up,out_channels=out_channels,kernel_size=3,stride=1,padding=1)self.conv_down = nn.Conv2D(in_channels=in_channels_down,out_channels=out_channels,kernel_size=3,stride=1,padding=1)self.conv_sub = nn.Conv2D(in_channels=out_channels,out_channels=out_channels_sub,kernel_size=3,stride=1,padding=1)self.relu = nn.ReLU()self.upsample = nn.PixelShuffle(upscale_factor=2)def forward(self, up: paddle.Tensor, down: paddle.Tensor) -> paddle.Tensor:up = self.conv_up(up)up = self.relu(up)down = self.conv_down(down)down = self.relu(down)sub = self.conv_sub(down)sub_up = self.upsample(sub)return paddle.concat([up, sub_up], axis=1)

CED 模型主体

堆叠五个层级的 CED Block，从高层级特征至低层级特征反向逐层细化边缘特征
最后使用一个卷积层作为模型的输出层计算最终的边缘结果

class CEDHead(nn.Layer):def __init__(self,fea_channels: List[int] = [64, 128, 256, 512, 512],num_classes: int = 1) -> None:super().__init__()self.head_blocks = nn.LayerList()for down_channels, up_channels in zip(fea_channels[:0:-1], fea_channels[-2::-1]):block = CEDBlock(in_channels_up=up_channels,in_channels_down=down_channels,out_channels_sub=up_channels*2,out_channels=up_channels//2)self.head_blocks.append(block)self.head_output = nn.Conv2D(in_channels=fea_channels[0],out_channels=num_classes,kernel_size=3,stride=1,padding=1)def forward(self, fea_inputs: List[paddle.Tensor]) -> List[paddle.Tensor]:fea_inputs = fea_inputs[::-1]down_feature = fea_inputs[0]for i, block_layer in enumerate(self.head_blocks):down_feature = block_layer(fea_inputs[i+1], down_feature)return [self.head_output(down_feature)]

CED

class CED(nn.Layer):def __init__(self,backbone: nn.Layer = vgg16(),backbone_indices: List[int] = [1, 3, 6, 9, 12],num_classes: int = 1) -> None:super().__init__()self.backbone = backboneself.backbone_indices = backbone_indicesbackbone_channels = [backbone.feat_channels[i]for i in backbone_indices]self.head = CEDHead(fea_channels=backbone_channels,num_classes=num_classes)def forward(self, inputs: paddle.Tensor) -> List[paddle.Tensor]:feat_list = self.backbone(inputs)feat_list = [feat_list[i] for i in self.backbone_indices]outputs = self.head(feat_list)return outputs

模型测试

model = CED()out = model(paddle.randn((1, 3, 224, 224))))(out[0]).shape

[1, 1, 224, 224]

总结

介绍了一下 CED 模型的主要改进，实现了 CED 模型的代码
不过由于官方没有提供预训练模型，训练的代码也暂时没有迁移完成（咕咕咕），所有目前只有模型的代码实现

边缘检测系列5：【CED】添加了反向细化路径的 HED 模型相关推荐

python中气泡图文字标签_Excel中制作气泡图及为气泡图的系列数据点添加文本数据标签...
Excel中制作气泡图及为气泡图的系列数据点添加文本数据标签时间:2014-08-01 作者:snow 来源:互联网 Excel中的散点图可以显示两组数据之间的关系,而气泡图则可以显示三组数 ...
战舰V3适配oneos系列03：添加SD卡驱动及文件系统
战舰V3系列03:添加SD卡驱动及文件系统本系列以 oneos2.3.0 提供的 STM32F103ZE 模板为基础,将 oneos 在战舰 V3 上运行起来,并逐步适配相关外设,计划周更本系列相 ...
战舰V3适配oneos系列05：添加外部SRAM
战舰V3系列05:添加外部SRAM 本系列以 oneos2.3.0 提供的 STM32F103ZE 模板为基础,将 oneos 在战舰 V3 上运行起来,并逐步适配相关外设,计划周更本系列相关代码将 ...
战舰V3适配oneos系列02：添加串口驱动
战舰V3适配oneos系列02:添加串口驱动参考: https://os.iot.10086.cn/v2/doc/detailPage/documentHtml?idss=1574615317503 ...
边缘检测系列4：【RCF】基于更丰富的卷积特征的边缘检测
引入上一篇介绍了经典的 HED 边缘检测模型这一次继续介绍另一篇边缘检测方向的经典论文:Richer Convolutional Features for Edge Detection 其中提出了 ...
边缘检测系列3：【HED】 Holistically-Nested 边缘检测
引入除了传统的边缘检测算法,当然也有基于深度学习的边缘检测模型这次就介绍一篇比较经典的论文 Holistically-Nested Edge Detection 其中的 Holistically- ...
ONNX系列六 --- 在Java中使用可移植的ONNX AI模型
目录安装和导入ONNX运行时载入ONNX模型使用ONNX运行时进行预测摘要和后续步骤参考文献下载源547.1 KB 系列文章列表如下: ONNX系列一 --- 带有ONNX的便携式神经网络 ...
深度学习系列笔记——贰（基于Tensorflow2 Keras搭建的猫狗大战模型三）
深度学习系列笔记--贰 (基于Tensorflow Keras搭建的猫狗大战模型一) 深度学习系列笔记--贰 (基于Tensorflow Keras搭建的猫狗大战模型二) 前面两篇博文已经介绍了如 ...

边缘检测系列5：【CED】添加了反向细化路径的 HED 模型

引入

参考资料

效果参考

模型架构

PixelShuffle

代码实现

骨干网络

CED 模块

CED 模型主体

CED

模型测试

总结

边缘检测系列5：【CED】添加了反向细化路径的 HED 模型相关推荐

最新文章

热门文章