How can just wearing a specific type of t-shirt make you invisible to the person detection and human surveillance systems? Well, researchers have found and exploited the Achilles’ heel of deep neural networks — the framework behind some of the best object detectors out there (YOLOv2, Faster R-CNN, HRNetv2, to name a few).

仅穿着特定类型的T恤如何使人检测和人类监视系统看不见？ 好吧，研究人员已经发现并利用了深度神经网络的致命弱点-深陷其中的一些最佳对象检测器(YOLOv2，Faster R-CNN，HRNetv2等)。

较早的方法： (Earlier approach:)

In [1], the authors manage to get a benchmark accuracy of deception of 57% in real-world use cases. However, this is not the first time attempts have been made to deceive an object detector. In [2] the authors designed a way for their model to learn and generate patches that could deceive the detector. This patch, when worn on a cardboard piece (or any flat surface) could evade the person detector albeit with an accuracy of 18%

在[1]中，作者设法在实际使用案例中获得了57％的基准欺骗准确性。 但是，这并不是第一次尝试欺骗对象检测器。 在[2]中，作者为他们的模型设计了一种方法来学习并生成可能欺骗检测器的补丁。 将该贴片戴在硬纸板片(或任何平坦表面)上时，即使准确度为18％，也可以避开人体检测仪

“Confusing” or “fooling” the neural network like this is called making a physical adversarial attack or a real-world adversarial attack. These attacks, initially based on intricately altered pixel values, confuse the network (based on its training data) into labeling the object as “unknown” or simply ignoring it.

像这样“混淆”或“欺骗”神经网络称为进行物理对抗攻击或真实世界对抗攻击。 这些攻击最初基于复杂变化的像素值，使网络(基于其训练数据)使该对象标记为“未知”，或者只是忽略了它。

Authors in [2] transform images in their training data, apply an initial patch, and feed the resulting image into the detector. The object loss obtained is used to change the pixel values in the patch and aimed at minimising the objectness score.

[2]中的作者将其训练数据中的图像进行转换，应用初始补丁，然后将生成的图像输入检测器。 获得的对象损失是用来改变在补丁中的像素值和旨在最小化对象性得分。

However, other than the low accuracy of 18%, this approach is limited to rigid carriers like a cardboard and doesn’t perform well when the captured frame has a distorted or skewed patch. Moreover, it certainly doesn’t work well when printed on t-shirts.

但是，除了18％的低精度外，此方法仅限于硬纸板之类的刚性载体，并且当捕获的框架变形或倾斜时，效果不佳。 而且，当印在T恤上时，它当然不能很好地工作。

“A person’s movement can result in significantly and constantly changing wrinkles (aka deformations) in their clothes” [1]. Thus making the task of developing a generalised adversarial patch even more difficult.

“一个人的运动可能会导致其衣服中的皱纹持续显着变化(又称变形)” [1]。 因此，开发通用对抗补丁的任务变得更加困难。

新的方法： (New Approach:)

The new approach in [1] employs Thin Plate Spline Mapping to model cloth deformations. These deformations simulate a realistic problem faced by previous attempts at using adversarial patterns. Taking care of different deformations would drastically improve the system's performance as it would be able to not-detect the pattern in more number of frames.

[1]中的新方法采用薄板样条映射来模拟布料变形。 这些变形模拟了以前使用对抗性模式所面临的现实问题。 照顾不同的变形将极大地改善系统的性能，因为它将无法在更多帧中检测到图案。

Understanding Splines themselves would be enough to get a rough idea of what they are trying to do with this approach.

理解样条线本身就足以大致了解他们要使用此方法进行的操作。

花键： (Splines:)

For a more formal, mathematical definition you can check this out, and for a more simplified understanding, I think this article does it best.

对于一个比较正式的，数学定义你可以检查这个出来，一个更简单的理解，我觉得这个文章做它最好的。

In an intuitive sense, splines help plot arbitrary functions smoothly — especially ones that require interpolations. Splines help model this missing data: here in modeling cloth deformation, where deformations in the patch shape can be seen in successive frames, we can use an advanced form of polynomial splines called Thin Plate Spline (TPS).

从直觉上讲，样条曲线有助于平滑绘制任意函数，尤其是那些需要插值的函数。 样条线有助于对缺失的数据进行建模：这里是在布料变形建模中，在连续的帧中可以看到补丁形状的变形，我们可以使用称为薄板样条线 (TPS)的多项式样条线的高级形式。

Check out this article by Columbia that illustrates and explains TPS Regression well.

查看Columbia 撰写的这篇文章 ，它很好地说明和解释了TPS回归。

These changes, or displacements, in the patch frames overtime are then modeled simply as a regression problem (since we only need to predict the TPS parameters for future frames).

然后，将补丁帧超时中的这些变化或位移简单地建模为回归问题(因为我们只需要预测未来帧的TPS参数)。

生成T恤图案： (Generating the T-shirt Pattern:)

The said pattern is just an adversarial example — a patch that acts against the purpose of the object detector. The authors use the Expectation Over Transformation (EOT) algorithm which helps in generating such adversarial examples over a given transformation distribution.

所述模式仅是一个对抗性示例-违反目标检测器目的的补丁。 作者使用转换期望(EOT)算法 ，该算法有助于在给定的转换分布上生成此类对抗性示例。

Here, the transformation distribution is made up of the TPS transformations since we want to replicate the real-time wrinkling, minor twisting, and changes in the contours of the fabric.

在这里，变换分布由TPS变换组成，因为我们要复制实时起皱，较小的扭曲以及织物轮廓的变化。

Along with TPS transformation they also use physical color transformation and conventional physical transformation within the person’s bounding box. Thus, this gives rise to the equation that models pixel values for the perturbed image.

除了TPS转换，他们还使用人的边界框内的物理颜色转换和常规物理转换。 因此，这引起了为被扰动的图像建模像素值的方程式。

The EOT formulation based on all these complex formulations can finally compute the attack loss and work towards fooling the object detector.

基于所有这些复杂公式的EOT公式最终可以计算出攻击损失并努力欺骗对象检测器。

The explanation of the procedure, in its most simplified form, so far is for single object detectors. The authors have also proposed a strategy for multiple object detectors that involves applying min-max optimization to the single object detector equation.

迄今为止，该过程以其最简化的形式针对单个对象检测器进行了说明。 作者还提出了一种用于多目标检测器的策略，该策略涉及将最小-最大优化应用于单个目标检测器方程。

最后： (Finally:)

The results after training and testing on their own dataset are impressive.

经过对自己的数据集进行训练和测试后，结果令人印象深刻。

And the use of TPS shows great improvement too:

TPS的使用也显示出巨大的改进：

未来是什么样子的： (What the future holds:)

• In an article by the Northeastern University, Xue Lin, one of the authors of [1], clarified that their goal isn’t to create a T-shirt in order to furtively go unnoticed by the detectors.

  [1]的作者之一薛林在东北大学的一篇文章中澄清说，他们的目标不是制造T恤以偷偷摸摸地被探测器发现。

“The ultimate goal of our research is to design secure deep-learning systems, … But the first step is to benchmark their vulnerabilities.” — Xue Lin

“我们研究的最终目标是设计安全的深度学习系统，但是，第一步是对它们的漏洞进行基准测试。” 薛林

• Certainly the authors realise the great scope of improvement in their results and mention that further research will be done to achieve it.当然，作者意识到结果的巨大改进范围，并提到将进行进一步的研究以实现这一目标。

Thank you for reading all the way through! You can reach out to me on LinkedIn for any messages, thoughts, or suggestions.

感谢您一直阅读！ 您可以在LinkedIn上与我联系，以获取任何消息，想法或建议。