【论文阅读】Graphonomy: Universal Human Parsing via Graph Transfer Learning通过图迁移学习进行的通用人体解析

Problem问题

人体解析是指将在图像中捕获的人分割成多个语义上一致的区域，例如，身体部位和衣物。作为一种细粒度的语义分割任务，它比仅是寻找人体轮廓的人物分割更具挑战性。人体解析对于以人为中心的分析非常重要，并且具有许多工业上的应用，例如，虚拟现实，视频监控和人类行为分析等等。
Prior highly-tuned human parsing models tend to fit towards each dataset in a specific domain or with discrepant label granularity, and can hardly be adapted to other human parsing tasks without extensive re-training以前的调优后的人体解析模型往往适合特定领域内的每个数据集，或者具有不一致的标签粒度，如果没有重新训练，就很难适应其他人体解析任务。

Introduction 引言

The most straightforward solution to universal human parsing would be posing it as a multi-task learning problem, and integrating multiple segmentation branches upon one shared backbone network.之前通用的人体解析最直接的解决方案是将其作为一个多任务学习问题，并将多个分割分支集成到一个网络上。
1、This line of research only considers the brute-force feature-level information sharing while disregarding the underlying common semantic knowledge, such as label hierarchy, label visual similarity, and linguistic/context correlations. 1、这种研究只考虑了特征级的信息共享，而忽略了底层相同的语义知识，如标签层次结构、标签视觉相似性、语言/语境相关性等。
2、Some techniques are explored to capture the human structure information by resorting to complex graphical models However, they did not explicitly model the semantic correlations of different body parts and clothing accessories, and still show unsatisfactory results for rare fine-grained labels. 2、探索了通过复杂的图模型来获取人体结构信息的方法，但是没有明确的对不同身体部位和服装配饰的语义相关性进行建模，对于罕见的细粒度标签仍然表现出不令人满意的结果。
Challenge: discovering underlying semantic structures among different label granularity, performing proper transfer learning across different image domains, and identifying and utilizing label redundancies across related tasks. 在不同的标签粒度中发现潜在的语义结构，在不同的图像域中执行适当的转移学习，并在相关任务中识别和利用标签冗余。
In this paper, we aim to learn a single universal human parsing model that can tackle all kinds of human parsing needs by unifying label annotations from different domains or at various levels of granularity.在本文中，我们的目标是学习一个通用的人体解析模型，该模型通过统一来自不同领域或不同粒度级别的标签注释来处理各种人体解析任务。
One key factor of designing a universal human parsing agent is to have proper transfer learning and knowledge integration among different human parsing tasks, as the label discrepancy across different datasets设计通用人体解析模型的一个关键是在不同的人体解析任务之间进行适当的迁移学习和知识集成，因为不同数据集之间的标签存在差异

Contribution贡献

We make the first attempts to tackle all levels of human parsing tasks using a single universal model. 我们首次尝试使用通用模型来处理所有级别的人体解析任务。
We explore various graph transfer dependencies to enrich graph transfer capability, which enables our Graphonomy to distill universal semantic graph representation and enhance individualized representation for each label graph探索了各种图的传输依赖关系，丰富了图的传输能力，使得我们的模型能够提取出通用的语义图表示，增强了每个标签的图的独立表示
We demonstrate the effectiveness of Graphonomy on universal human parsing, showing that it achieves the state-of-the-art results on three human parsing datasets. 证明了Graphonomy的有效性在人体解析方面在三个数据集上取得最先进的结果.

Method方法

In particular, Graphonomy first learns and propagates compact high-level graph representation among the labels within one dataset via Intra-Graph Reasoning. Graphonomy首先通过图内推理在一个数据集内的标签之间学习和传播紧凑的高级图表示。
Then transfers semantic information across multiple datasets via Inter-Graph Transfer. Various graph transfer dependencies (e.g., similarity, linguistic knowledge) between different datasets are analyzed and encoded to enhance graph transfer capability.然后通过图间传输在多个数据集之间传输语义信息。分析和编码不同数据集之间的各种图数据传输依赖关系(如相似性、语言知识)，以增强图数据传输能力。
By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity.通过将通用语义图表示提取到每个特定任务中，Graphonomy能够在不增加复杂性的情况下预测一个系统中所有层次的标签。

Graphonomy

首先通过图内推理在一个数据集中学习和传播紧凑的高级语义图表示，然后通过显式的层次语义标签结构驱动的图间迁移，在多个数据集中传输和融合语义信息。

Intra-Graph Reasoning 图内推理

将feature映射为图表示

Formally, we use the feature maps X [C,H,W] as the module inputs, where H, W and C are height, width and channel number of the feature maps.
We first produce high-level graph representation Z[N,D] of all N vertices, where D is the desired feature dimension.我们首先生成所有N个顶点的高级图表示Z[N,D]，其中D是所需的特征维数。

三次图卷积过程

Where We is a trainable weight matrix and σ is a nonlinear function.
W是可训练权重矩阵，σ是一个非线性激活函数。
The node adjacency weight

【论文阅读】Graphonomy: Universal Human Parsing via Graph Transfer Learning通过图迁移学习进行的通用人体解析相关推荐

论文阅读【CVPR-2022】 A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation
论文阅读 [CVPR-2022] A Simple Multi-Modality Transfer Learning Baseline for Sign Language Translation st ...
Graph Structure Learning（图结构学习应用续篇）
博主在以往的文章中更新过图结构学习的相关概念,和北邮团队的几篇关于图结构学习的文章(主要KDD20,AAAI21,WWW21,AAAI21). Graph Structure Learning(图结构 ...
论文阅读：Natural Language Processing Advancements By Deep Learning: A Survey 深度学习在自然语言处理中的进展
Natural Language Processing Advancements By Deep Learning: A Survey 深度学习在自然语言处理中的进展目录 Natural Langu ...
【论文笔记09】Differentially Private Hypothesis Transfer Learning 差分隐私迁移学习模型, ECMLPKDD 2018
目录导引系列传送 Differentially Private Hypothesis Transfer Learning 1 Abstract 2 Bg & Rw 3 Setting &am ...
论文阅读：Natural Language Processing Advancements By Deep Learning: A Survey
文章目录一.介绍二.背景 1.人工智能和深度学习 (1)多层感知机 (2)卷积神经网络 (3)循环神经网络 (4)自编码器 (5)生成对抗网络 2.NLP中深度学习的动机三.NLP领域的核心概念 ...
【论文阅读】A Gentle Introduction to Graph Neural Networks [图神经网络入门]（7）
[论文阅读]A Gentle Introduction to Graph Neural Networks [图神经网络入门](7) Into the Weeds Other types of grap ...
【论文阅读】A Gentle Introduction to Graph Neural Networks [图神经网络入门]（6）
[论文阅读]A Gentle Introduction to Graph Neural Networks [图神经网络入门](6) GNN playground Some empirical GNN ...
【论文阅读】A Gentle Introduction to Graph Neural Networks [图神经网络入门]（5）
[论文阅读]A Gentle Introduction to Graph Neural Networks [图神经网络入门](5) Graph Neural Networks 图神经网络 Now th ...
【论文阅读】A Gentle Introduction to Graph Neural Networks [图神经网络入门]（4）
[论文阅读]A Gentle Introduction to Graph Neural Networks [图神经网络入门](4) The challenges of using graphs in ...

【论文阅读】Graphonomy: Universal Human Parsing via Graph Transfer Learning通过图迁移学习进行的通用人体解析

Problem问题

Introduction 引言

Contribution贡献

Method方法

Graphonomy

Intra-Graph Reasoning 图内推理

将feature映射为图表示

三次图卷积过程

【论文阅读】Graphonomy: Universal Human Parsing via Graph Transfer Learning通过图迁移学习进行的通用人体解析相关推荐

最新文章

热门文章