FSCE：通过对比建议编码进行少样本目标检测

https://github.com/MegviiDetection/FSCE

Emerging interests have been brought to recognize previously unseen objects given very few training examples, known as few-shot object detection (FSOD). Recent researches demonstrate that good feature embedding is the key to reach favorable few-shot learning performance.We observe object proposals with different Intersection-ofUnion (IoU) scores are analogous to the intra-image augmentation used in contrastive approaches. And we exploit this analogy and incorporate supervised contrastive learning to achieve more robust objects representations in FSOD.

新出现的兴趣已被用于识别以前看不见的对象，因为训练示例很少，称为少镜头对象检测 (FSOD)。最近的研究表明，良好的特征嵌入是获得良好的少样本学习性能的关键。我们观察到具有不同 Intersection-ofUnion (IoU) 分数的对象提议类似于对比方法中使用的图像内增强。我们利用这种类比并结合监督对比学习，在 FSOD 中实现更稳健的对象表示。

We present Few-Shot object detection via Contrastive proposals Encoding (FSCE), a simple yet effective approach to learning contrastive-aware object proposal encodings that facilitate the classification of detected objects. We notice the degradation of average precision (AP) for rare objects mainly comes from misclassifying novel instances as confusable classes. And we ease the misclassification issues by promoting instance level intra-class compactness and interclass variance via our contrastive proposal encoding loss (CPE loss). Our design outperforms current state-of-theart works in any shot and all data splits, with up to +8:8% on standard benchmark PASCAL VOC and +2:7% on challenging COCO benchmark.

我们通过对比提议编码 (FSCE) 提出 Few-Shot 对象检测，这是一种简单而有效的学习对比感知对象提议编码的方法，有助于对检测到的对象进行分类。我们注意到稀有对象的平均精度（AP）的下降主要来自于将新实例错误分类为易混淆的类。我们通过对比提议编码损失（CPE 损失）提高实例级别的类内紧凑性和类间方差，从而缓解错误分类问题。 我们的设计在任何镜头和所有数据拆分中都优于当前的最新工作，在标准基准 PASCAL VOC 上高达 +8:8%，在具有挑战性的 COCO 基准上 +2:7%。

1. Introduction

Development of modern convolutional neural networks (CNNs) [1, 2, 3] give rise to great advances in general object detection [4, 5, 6]. Deep detectors demand a large amount of annotated training data to saturate its performance [7, 8].
In few-shot learning scenarios, deep detectors suffer severer over-fitting and the gap between few-shot detection and general object detection is larger than the corresponding gap in few-shot image classification [9, 10, 11]. On the contrary,a child can rapidly comprehend new visual concepts and recognize objects from a newly learned category given very few examples. Closing such gap is therefore an important step towards more successful machine perception [12].

现代卷积神经网络 (CNN) [1, 2, 3] 的发展在一般目标检测 [4, 5, 6] 方面取得了巨大进步。深度检测器需要大量带注释的训练数据才能使其性能饱和 [7, 8]。在few-shot学习场景中，深度检测器遭受更严重的过拟合，few-shot检测与一般目标检测之间的差距大于few-shot图像分类中相应的差距[9,10,11]。相反，孩子可以快速理解新的视觉概念，并通过很少的例子从新学习的类别中识别对象。因此，缩小这种差距是朝着更成功的机器感知迈出的重要一步[12]。

Precedented by few-shot image classification, earlier attempts in few-shot object detection utilize meta-learning strategy [13, 14, 15]. Meta-learners are trained with an episode of individual tasks, meta-task samples from common objects (base class) to pair with rare objects (novel class) to simulate few-shot detection tasks. Recently, the two-stage fine-tune based approach (TFA) reveals more potential in improving few-shot detection. Baseline TFA [16] simply freeze all base class trained parameters and fine-tune only box classifier and box regressor with novel data, yet outperforms previous meta-learners. MPSR [17] improves upon TFA by alleviating the scale bias inherent to few-shot dataset, but their positive refinement branch demands manual selection, which is somewhat less neat. In this work, we observe and address the essential weakness of the finetuning based approach – constantly mislabeling novel instances as confusable categories, and improve the few-shot detection performance to the new state-of-the-art (SOTA).

在少镜头图像分类之前，早先的少镜头目标检测尝试利用元学习策略[13,14,15]。元学习者接受一系列单独任务的训练，来自常见对象（基类）的元任务样本与稀有对象（新类）配对以模拟小样本检测任务。最近，基于两阶段微调的方法 (TFA) 在改进少样本检测方面显示出更大的潜力。基线 TFA [16] 简单地冻结所有基类训练的参数并仅用新数据微调框分类器和框回归器，但优于以前的元学习器。 MPSR [17] 通过减轻少样本数据集固有的尺度偏差来改进 TFA，但它们的正细化分支需要手动选择，这有点不那么整洁。在这项工作中，我们观察并解决了基于微调的方法的基本弱点——不断将新实例错误地标记为易混淆的类别，并将少数样本检测性能提高到新的最先进技术 (SOTA)。

Object detection involves localization and classification of appeared objects. In few-shot detection, one might naturally conjecture the localization of novel objects is going to under-perform its base categories counterpart, with the concern that rare objects would be deemed as background [14, 13, 18]. However, based on our experiments with Faster R-CNN [4], the commonly adopted detector in few-shot detection, class-agonistic region proposal network (RPN) is able to make foreground proposals for novel instances, and the final box regressor can localize novel instances quite accurately. In comparison, as demonstrated in Figure 2, misclassifying detected novel instances as confusable base classes is indeed the main source of error. We visualize the pairwise cosine similarity between class prototypes [19, 20, 21] of a Faster R-CNN box classifier trained with PASCAL VOC [22, 23]. The cosine similarity between prototypes from resembled categories can be 0:39, whereas the similarity between objects and background is on average -0.21. In few-shot setting, the similarity between cluster centers can go as high as 0.59, e.g., between sheep and cow, bicycle and motorbike, making classification for similar objects error-prone. We make a calculation upon baseline TFA, manually correcting misclassified yet accurately localized box predictions can increase novel class average precision (nAP) by over 20 points.

对象检测涉及出现对象的定位和分类。在少镜头检测中，人们可能会自然地推测新物体的定位将低于其基本类别对应物，担心稀有物体会被视为背景 [14,13,18]。然而，基于我们对 Faster R-CNN [4] 的实验，在少样本检测中常用的检测器、类竞争区域提议网络 (RPN) 能够为新实例提出前景提议，并且最终的框回归器可以非常准确地定位新实例。相比之下，如图 2 所示，将检测到的新实例错误分类为可混淆的基类确实是主要的错误来源。我们可视化使用 PASCAL VOC [22, 23] 训练的 Faster R-CNN 盒分类器的类原型 [19, 20, 21] 之间的成对余弦相似度。来自相似类别的原型之间的余弦相似度可以为 0:39，而对象和背景之间的相似度平均为 -0.21。在few-shot设置中，聚类中心之间的相似度可以高达0.59，例如羊和牛、自行车和摩托车之间，使得相似对象的分类容易出错。我们基于基线 TFA 进行计算，手动纠正错误分类但准确定位的框预测可以将新的类平均精度 (nAP) 提高 20 多个点。

Figure 2. We find in fine-tuning based few-shot object detector, classification is more error-prone than localization. In the fine-tuning stage, RPN is able to make good enough foreground proposals for novel instances, hence novel objects are often accurately localized but mis-classified as confusable base classes. Here shows 20 top-scoring RPN proposals and example detection results from PASCAL VOC Split 1, wherein bird, sofa and cow are novel categories. The left panel shows the pair-wise cosine similarity between the class prototypes learned in the bounding box classifier. For example, the similarity between bus and bird is -0.10, but the similarity between cow and horse is 0.39. Our goal is to decrease the instance-level similarity between similar objects that are from different categories.

图 2. 我们发现在基于微调的小样本目标检测器中，分类比定位更容易出错。在微调阶段，RPN 能够为新实例提供足够好的前景建议，因此新对象通常被准确定位但被错误分类为可混淆的基类。这里展示了来自 PASCAL VOC Split 1 的 20 个得分最高的 RPN 提议和示例检测结果，其中鸟、沙发和牛是新类别。左图显示了在边界框分类器中学习的类原型之间的成对余弦相似度。比如bus和bird的相似度是-0.10，但是cow和horse的相似度是0.39。我们的目标是降低来自不同类别的相似对象之间的实例级相似性。

A common approach to learn well-separated decision boundary is to use a large margin classifier [24], but with our trials, category-level positive-margin based classifiers does not work in this data-hunger setting [20, 25]. To learn instance-level discriminative feature representations, contrastive learning [26, 27] has demonstrated its effectiveness in tasks including recognition [28], identification [29] and the recent successful self-supervised models [30, 31, 32, 33]. In supervised contrastive learning for image classification [34], intra-image augmentations of images from the same class are used to enrich the positive example pairs.We think region proposals with different Intersection-overUnion (IoU) for an object are naturally analogous to the intra-image augmentation cropping, as illustrated in Figure 1. Therefore in this work, we explore to extend the supervised batch contrastive approach [34] to few-shot object detection. We believe the contrastively learned object representations aware of the intra-class compactness and the inter-class difference can ease the misclassification of unseen objects as similar categories.

学习分离良好的决策边界的一种常用方法是使用大边距分类器 [24]，但在我们的试验中，基于类别级正边距的分类器在这种数据饥饿环境中不起作用 [20, 25]。为了学习实例级的判别特征表示，对比学习 [26, 27] 已经证明了它在识别 [28]、识别 [29] 和最近成功的自监督模型 [30, 31, 32, 33] 等任务中的有效性。在图像分类的监督对比学习[34]中，来自同一类的图像的图像内增强被用来丰富正样本对。 我们认为对于一个对象具有不同 Intersection-overUnion (IoU) 的区域提议自然类似于图像内增强裁剪，如图 1 所示。因此，在这项工作中，我们探索将监督批量对比方法 [34] 扩展到少镜头目标检测。我们相信对比学习的对象表示意识到类内紧凑性和类间差异可以缓解将看不见的对象误分类为相似类别的问题。

Figure 1. Conceptualization of our contrastive object proposals encoding. We introduce a score function which measures the semantic similarity between region proposals. Positive proposals (x+) refer to region proposals from the same category or the same object. Negative proposals (

FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding相关推荐

1. 【论文总结】FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding（附翻译）

   论文地址:https://arxiv.org/pdf/2103.05950.pdf 代码地址:https: //github.com/MegviiDetection/FSCE 改进:主要是针对小样本检 ...

2. 【代码调试】《FSCE: Few-Shot Object Detection via Contrastive Proposal Encoding》

   论文地址:https://arxiv.org/pdf/2103.05950.pdf 代码地址:https://github.com/megvii-research/FSCE 论文阅读:https:// ...

3. 深度学习论文阅读目标检测篇（三）：Faster R-CNN《 Towards Real-Time Object Detection with Region Proposal Networks》

   深度学习论文阅读目标检测篇(三):Faster R-CNN< Towards Real-Time Object Detection with Region Proposal Networks&g ...

4. 目标检测经典论文——Faster R-CNN论文翻译：Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Net

   Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks Faster R-CNN:通过Region ...

5. 读书笔记：Faster R-CNN：Towards Real-Time Object Detection with Region Proposal Networks

   Faster R-CNN:Towards Real-Time Object Detection with Region Proposal Networks 著名的faster R-CNN 当然应该好好 ...

6. Single shot object detection SSD using MobileNet and OpenCV

   微信公众号:小白图像与视觉 关于技术.关注yysilence00.有问题或建议,请公众号留言. 主题:Single shot object detection SSD using MobileNet ...

7. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks(更快的RCNN：通过区域提议网络实现实时)

   原文出处 感谢作者~ Faster R-CNN: Towards Real-Time Object Detection with Region ProposalNetworks Shaoqing Re ...

8. 《Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks》论文翻译

   原文:https://arxiv.org/abs/1506.01497 Faster R-CNN: Towards Real-Time ObjectDetection with Region Prop ...

9. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks（理解）

   0 - 背景 R-CNN中检测步骤分成很多步骤,fast-RCNN便基于此进行改进,将region proposals的特征提取融合成共享卷积层问题,但是,fast-RCNN仍然采用了selectiv ...

10. Faster R-CNN 《Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks》论文笔记

    本学弱喜欢在本子上记笔记,但字迹又丑. 望看不懂我的字的大佬不要喷我,看得懂的大佬批评指正.

最新文章

1. Science：发现重写创伤记忆的神经元
2. python打砖块游戏算法设计分析_python实现打砖块游戏
3. 从动力学角度看优化算法：为什么学习率不宜过小？
4. IBatis.Net学习笔记十三：在IBatis.Net中调用存储过程
5. js在控件原有的事件方法中加入自己的方法
6. Effective Java之坚持使用Overide注解(三十六)
7. 成为0.01%！利用TensorFlow.js和深度学习，轻松阅读古草体文字
8. 代码整洁之道1-6章总结
9. ubuntu18.04使用calibration_toolkit进行相机和三维激光雷达的联合标定
10. 从百亿美元到零估值，硅谷的Theranos或许不止一个
11. Android逆向:去除RE管理器4.41及车来了广告
12. 【C语言】求一个数有多少位
13. SVN(Subversion)中文版安装教程
14. [转]计算机程序设计的史诗TAOCP
15. LightGBM 参数及调优
16. wine葡萄酒数据集KNNSVM分类实验
17. 通往幸福之路之贷款篇
18. JDBC数据库连接操作
19. 轻松可视化实现设备监控大屏效果
20. 微信公众号自定义回复（关键词回复:文本消息、图文消息）

热门文章

1. 董明珠揭示：未来2年这个行业盈利最大，马总点赞说，又要出富翁
2. Python爬取校花网
3. dytt 爬取磁力链接保存mysql
4. 软件包的依赖性关系定义
5. css 三栏布局 圣杯布局 双飞翼 flex
6. JavaScript模板引擎-artTemplate
7. 实习期间的一些思考整理（2）2018.4.10~4.11
8. Apache jmeter 压测
9. Python基础之面向对象基础
10. ue4 FString 中文乱码问题