【SeMask】Semantically Masked Transformers for Semantic Segmentation

SeMask：Semantically Masked Transformers for Semantic Segmentation

【Paper】【Github】

问题：encoding阶段（基于Transformer主干），语义信息是有所丢失的。

Use an existing pretrained backbone as an encoder and transfer it to downstream tasks using pre-existing standard decoders such as, Semantic FPN or UperNet .利用现存的与训练主干作为encoder进行下游任务。
design a new encoder-decoder network where the encoder is pretrained on ImageNet for the semantic segmentation task.设计新的encoder-decoder，其中encoder是预训练的。
从大规模数据集进行微调有助于早期注意层在transformer的底层合并局部信息。 但是， 由于数据集的规模相对较小，以及语义类的数量和性质从分类到分割任务的变化，它在微调过程中仍然不能利用语义上下文。
语义上下文信息的缺乏导致分割性能次优，特别是在小对象的情况下，这些对象与大类别的边界合并，导致错误的预测。

方案：incorporating，将语义信息整合

结构：

SeMask， incorporates semantic information into the encoder with the help of a semantic attention operation.
SeMask uses semantic priors in the encoder backbones by adding an additional semantic

layer; this simple change signifificantly improves performance.

具体：

Use a lightweight semantic decoder during training to provide supervision to the intermediate semantic prior maps at every stage.
Provide empirical proof by integrating SeMask into Swin Transformer and Mix Transformer backbones as our encoder paired with different decoders.

Method

笔记摘录，未细究。

【SeMask】Semantically Masked Transformers for Semantic Segmentation相关推荐

【FCN】Fully Convolutional Networks for Semantic Segmentation学习
论文翻译又是一个看不懂原文,又看不懂翻译的论文. semantic segmentation 图像语义分割就是预测每个像素所属的对象类别语义分割是指像素级地识别图像,即标注出图像中每个像素所属的 ...
【2015-CVPR】Fully Convolutional Networks for Semantic Segmentation
概述卷积网路自身逐像素端到端训练效果好于目前的SOTA语义分割网络,本文提出了FCN可接受任意大小输入并推理学习产生对应大小输出,它将来自深层的粗糙层语义信息和来自浅层的精细的外观信息相结合,该网络 ...
【FCN2015】Fully Convolutional Networks for Semantic Segmentation
Fully Convolutional Networks for Semantic Segmentation 用于语义分割的全卷积网络 arXiv:1411.4038v2 [cs.CV] 8 Mar ...
【深度学习】【CVPR2019 oral】Structured Knowledge Distillation for Semantic Segmentation
[CVPR2019 oral]Structured Knowledge Distillation for Semantic Segmentation KD Intro Approach Structu ...
【U-HRNet2022】U-HRNet: Delving into Improving Semantic Representation of High Resolution Network for
U-HRNet: Delving into Improving Semantic Representation of High Resolution Network for Dense Predict ...
【论文翻译】Fully Convolutional Networks for Semantic Segmentation
论文题目:Fully Convolutional Networks for Semantic Segmentation 论文来源:Fully Convolutional Networks for Se ...
【语义分割】DUC -- Understand Convolution for Semantic Segmentation
参考:Understand Convolution for Semantic Segmentation
【论文】3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation
论文题目:3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation 3D u-net,从稀疏注释中学习密集体分割 ...
【论文翻译】AFFormer：Head-Free Lightweight Semantic Segmentation with Linear Transformer
论文地址:https://arxiv.org/pdf/2301.04648v1.pdf Abstract 现有的语义分割工作主要集中在设计有效的解码器上:然而,整体结构引入的计算负载长期以来一直被 ...

【SeMask】Semantically Masked Transformers for Semantic Segmentation

【SeMask】Semantically Masked Transformers for Semantic Segmentation相关推荐

最新文章

热门文章