论文笔记：CVPR2021 Bottom-Up Shift and Reasoning for Referring Image Segmentation

任务名字：Referring Image Segmentation (RIS)

keywords：one-stage RIS、graph、relation reasoning

背景：方法比较

vision-and-language approaches based on their designing principles,

（1）multimodal fusion and representation learning

（2）language-conditioned visual rea- soning

two-stage RIS：

优：explicit object instances and their relation-ships to conduct visual reasoning

缺：slow inference speed 、has poor generalization、the relational and spatial priors in images are lost when conducting reasoning over feature vectors of those object instances.

one-stage RIS：

优：fast inference speed、contextual representations

缺：no ex-plicit object-level information、inferior in handling complex visual scenes and expressions because they lack sufficient visual reasoning capability

Method：

图像encoder：DeepLab ResNet101

language encoder: GloVe word embedding wt of each word l_t + position encoding

为了进一步增加词间相互关系的表达，引入了自注意力机制

Bottom-Up Shift:

（1）Analysis of Reasoning Steps

利用图表达，将复杂的推理抽象成简单的节点和边

使用language graph（directed acyclic graph）：A node and a directed edge of the graph respectively correspond to a noun phrase and the linguistic relationship

（2）Stepwise Inference逐步推理：

the reasoning from bottom to up

首先节点和图融合得到X

接下来，通过对节点之间的关系（即边）按照遍历的顺序进行逐步推理，将节点在图像中的初始空间位置转移到正确的位置。

同样，我们假设上的节点在当前步骤中作为节点处理。首先通过PRS对图的每条边单独执行关系推理，然后通过平均池操作集成所有连接边中节点o_n结果。

edges的集成，对于具有初始特征映射Xn和连接边En的节点，其更新的特征映射Xn′计算如下：

PRS（3）表示迭代三次

（3）Pairwise Relational Shift

Bidirectional Attentive Refinement:

将上个模块输出的x4、x5，与前文encoder的浅层特征v2、v3、v4通过自上而下的策略合并

因为浅层包含全图的详细信息，可能引起不相关的噪声，因此使用自注意力机制

最后上采样相加

论文笔记：CVPR2021 Bottom-Up Shift and Reasoning for Referring Image Segmentation相关推荐

【论文笔记】MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation
论文标题:MiniSeg: An Extremely Minimum Network for Efficient COVID-19 Segmentation 论文链接:https://arxiv.or ...
论文笔记——Rich feature hierarchies for accurate object detection and semantic segmentation
最近在看一些目标检测的论文,本文是经典的R-CNN(Regions with CNN features),随之产生的一系列目标检测算法:RCNN,Fast RCNN, Faster RCNN代表当下目 ...
论文笔记2.5D lightweight RIU-Net for automatic liver and tumor segmentation from CT
用于CT自动肝脏和肿瘤分割的2.5D轻量级RIU-Net ELSEVIER Biomedical Signal Processing and Control 2022 InceptionV1-V3 ...
[深度学习论文笔记]TransBTSV2: Wider Instead of Deeper Transformer for Medical Image Segmentation
TransBTSV2: Wider Instead of Deeper Transformer for Medical Image Segmentation TransBTSV2:用于医学图像分割的宽 ...
【论文笔记】Region-based Convolutional Networks for Accurate Object Detection and Segmentation
<Region-based Convolutional Networks for Accurate Object Detection and Segmentation>是将卷积神经网络应用 ...
医学论文笔记：TMI2021 SMU-Net: Saliency-Guided Morphology-Aware U-Net for Breast Lesion Segmentation ...
乳腺超声分割: 文章指出周围组织(即背景)和病变区域(即前景)之间的模式复杂性和强度相似性给病变分割带来了挑战.考虑到背景中包含如此丰富的纹理信息,很少有方法尝试探索和利用背景显着表示来辅助前景分割. ...
论文笔记 |【CVPR2021】Uformer: A General U-Shaped Transformer for Image Restoration
论文笔记 |[CVPR2021]Uformer: A General U-Shaped Transformer for Image Restoration 文章目录论文笔记 |[CVPR2021]U ...
论文笔记【A Comprehensive Study of Deep Video Action Recognition】
论文链接:A Comprehensive Study of Deep Video Action Recognition 目录 A Comprehensive Study of Deep Video A ...
【论文笔记】Encoding cloth manipulations using a graph of states and transitions
[论文笔记]Encoding cloth manipulations using a graph of states and transitions Abstract 问题: Cloth manipu ...
最新图神经网络论文笔记汇总（附pdf下载）
点击上方,选择星标或置顶,不定期资源大放送! 阅读大概需要15分钟 Follow小博主,每天更新前沿干货 [导读]近年来,图神经网络变得非常火热,每年顶会在该领域内都会出现大量的研究论文,本文为大家提 ...

论文笔记：CVPR2021 Bottom-Up Shift and Reasoning for Referring Image Segmentation

论文笔记：CVPR2021 Bottom-Up Shift and Reasoning for Referring Image Segmentation相关推荐

最新文章

热门文章