《Restricted Deformable Convolution based Road Scene Semantic Segmentation Using Surround View Camer》

一、文章解决的问题

This paper addresses 360-degree road scene semantic segmentation using surround view cameras.使用环视相机解决360°道路语义分割问题。

二、技术方案

First, in order to address large distortion problem in the fisheye images, Restricted Deformable Convolution (RDC) is proposed for semantic segmentation, which can effectively model geometric transformations by learning the shapes of convolutional filters conditioned on the input feature map. Second, in order to obtain a large-scale training set of surround view images, a novel method called zoom augmentation is proposed to transform conventional images to fisheye images. Finally, an RDC based semantic segmentation model is built; the model is trained for real-world surround view images through a multi-task learning architecture by combining real-world images with transformed images.

首先，为了解决鱼眼图像中的巨大图像失真，提出了受限制的可形变卷积（RDC）用于语义分割；其次为了获得大规模的环视图像训练数据集，提出了zoom增强方法来将传统图像转换为鱼眼图像。最后建立了基于RDC的语义分割模型。

三、创新点

First, the Restricted Deformable Convolution (RDC) is proposed to enhance the transformation modeling capability of CNNs, so that the net can handle the images with large distortions. Second, in order to enrich surround view training data which
are lacking, the zoom augmentation method is proposed to transform conventional images to fisheye images. Two existing
complementary datasets are transformed using this method. Finally, an RDC based semantic segmentation model is trained
for real-world surround view images through a multi-task learning architecture with the approaches of AdaBN and HLW.

首先，提出了受限可变形卷积(Restricted Deformable Convolution, RDC)算法，以增强CNNs的变换建模能力，使网络能够处理较大的畸变图像。其次，为了丰富缺少的环绕视图训练数据，提出了将传统图像转化为鱼眼图像的缩放增强方法。使用此方法转换两个现有的互补数据集。最后，采用AdaBN和HLW两种方法，通过多任务学习结构，对真实环境下的环绕视图图像进行基于RDC的语义分割模型训练。

四、实验方案与结果

We compare the proposed approach with fine-tuned FCNVGG16, ENet, and ERFNet on the test set of SVScape.
These models are fine-tuned from the pretrained weights on Cityscapes dataset. Table V shows per-class accuracy results.

五、可能存在的问题

RDC has a good ability to model geometric transformations and is less prone to saturation. Deformable convolution shows
a better ability of modeling geometric transformations if only applied to the last few convolutional layers. As future work, RDC and deformable convolution should be combined in one network to further enhance the CNNs’ transformation modeling ability. Future work also needs to incorporate weakly or other domain adaptation methods to further improve the performance on real surround view images.

RDC具有良好的几何变换建模能力，不易饱和。仅对最后几个卷积层应用可变形卷积，显示出更好的几何变换建模能力。在未来的工作中，RDC和可变形卷积应该结合在一个网络中，进一步增强CNNs的变换建模能力。未来的工作还需要结合弱监督或其他领域的适应方法，以进一步提高实际环绕视图图像的性能。

使用全景环视相机的基于RDC的道路语义分割阅读感悟相关推荐

笔记：基于DCNN的图像语义分割综述
写在前面:一篇魏云超博士的综述论文,完整题目为<基于DCNN的图像语义分割综述>,在这里选择性摘抄和理解,以加深自己印象,同时达到对近年来图像语义分割历史学习和了解的目的,博古才能通今!感 ...
深度学习（二十一）基于FCN的图像语义分割-CVPR 2015-未完待续
CNN应用之基于FCN的图像语义分割原文地址:http://blog.csdn.net/hjimce/article/details/50268555 作者:hjimce 一.相关理论本篇 ...
【深度学习】SETR：基于视觉 Transformer 的语义分割模型
Visual Transformer Author:louwill Machine Learning Lab 自从Transformer在视觉领域大火之后,一系列下游视觉任务应用研究也随之多了起来.基 ...
使用Pytorch搭建U-Net网络并基于DRIVE数据集训练(语义分割)学习笔记
使用Pytorch搭建U-Net网络并基于DRIVE数据集训练(语义分割)学习笔记 https://www.bilibili.com/video/BV1rq4y1w7xM?spm_id_from=33 ...
基于MindSpore复现Deeplabv3—语义分割
基于MindSpore复现Deeplabv3-语义分割实验介绍本实验主要介绍使用MindSpore深度学习框架在PASCAL VOC2012数据集上训练Deeplabv3网络模型.本实验使用了Mi ...
FEANet——基于 RGBT的实时语义分割特征增强注意力网络
Overview Title:FEANet: Feature-Enhanced Attention Network for RGB-Thermal Real-time Semantic Segment ...
基于迁移学习的语义分割算法分享与代码复现
摘要:语义分割的数据集是比较大的,因此训练的时候需要非常强大的硬件支持. 本文分享自华为云社区<[云驻共创]基于迁移学习的语义分割算法分享>,原文作者:启明. 此篇文章是分享两篇基于迁移学 ...
笔记：基于点云的语义分割的小样本学习
笔记:Few-shot learning for tackling open-set generalization: 基于点云的语义分割的应用:场景理解,给点云中每一个点赋予特点的语义标签.(如自动驾 ...
基于FCN的图像语义分割
语义图像分割的目标在于标记图片中每一个像素,并将每一个像素与其表示的类别对应起来.因为会预测图像中的每一个像素,所以一般将这样的任务称为密集预测.(相对地,实例分割模型是另一种不同的模型,该模型可以区 ...

使用全景环视相机的基于RDC的道路语义分割阅读感悟