有关深层学习的FAU讲义 (FAU LECTURE NOTES ON DEEP LEARNING)

These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. Try it yourself! If you spot mistakes, please let us know!

这些是FAU YouTube讲座“ 深度学习 ”的 讲义 。 这是演讲视频和匹配幻灯片的完整记录。 我们希望您喜欢这些视频。 当然，此成绩单是使用深度学习技术自动创建的，并且仅进行了较小的手动修改。 自己尝试！ 如果发现错误，请告诉我们！

导航 (Navigation)

Previous Lecture / Watch this Video / Top Level / Next Lecture

上一个讲座 / 观看此视频 / 顶级 / 下一个讲座

Welcome back to deep learning! Today, we want to talk about the last part of object detection and segmentation. We want to look into the concept of instance segmentation.

欢迎回到深度学习！ 今天，我们要讨论对象检测和分割的最后一部分。 我们想研究实例分割的概念。

So, let’s have a look at our slides. You see this is already the last part. Part five and now we want to talk about instance segmentation. We do not just want to detect where pixels with cubes are instead of pixels of cups. We want to really figure out which pixels belong to what cube. This is essentially a combination of object detection and semantic segmentation.

因此，让我们看一下幻灯片。 您已经看到这已经是最后一部分了。 第五部分，现在我们要讨论实例分段。 我们不只是要检测带有立方体的像素在哪里，而不是检测杯子的像素。 我们要真正找出哪些像素属于哪个立方体。 这实质上是对象检测和语义分割的组合。

Examples for potential applications are information about occlusion, counting the number of elements belonging to the same class, detecting object boundaries for example of gripping objects in robotics. This is very important and there are examples in the literature for simultaneous detection and segmentation, DeepMask, SharpMask, and Mask RCNN in [10].

潜在应用的示例包括有关遮挡的信息，计算属于同一类的元素的数量，检测对象边界(例如在机器人技术中抓取对象)。 这非常重要，文献[10]中提供了用于同时检测和分割的示例，DeepMask，SharpMask和Mask RCNN。

Let’s look at [10] in a little more detail. We essentially go back to the storage. We combine object detection and the segmentation. We use RCNN for object detection. It essentially solves the instance separation. Then, the segmentation refines the bounding boxes per instance.

让我们更详细地研究[10]。 我们实质上是回到存储。 我们结合了对象检测和分割。 我们使用RCNN进行对象检测。 它实质上解决了实例分离。 然后，分割会细化每个实例的边界框。

The workflow is a two-stage procedure. You have the region proposal that proposes the object bounding boxes. Then, you have the classification using a bounding box regression and the segmentation in parallel. So, you have a multi-task loss that essentially combines the pixel-classification loss of the segmentation, the box loss, and the class loss for producing the right class/bounding box. So, you have these three terms that are then combined in a multi-task loss.

工作流程分为两个阶段。 您具有提议对象边界框的区域提议。 然后，您可以使用包围盒回归和并行分割进行分类。 因此，您有一个多任务损失，该损失实际上将分割的像素分类损失，框损失和产生正确的类/边界框的类损失结合在一起。 因此，您具有这三个术语，然后将它们组合成多任务损失。

Let’s look in some more detail into the two-stage procedure. You have two different options here for two-stage networks. You can have a joint branch that is working on the ROIs and then splits at a later stage into the segmentation of the mask and the class and bounding box prediction, or you can split early. Then, you run that in two separate networks, In both versions, you have this multi-task loss and that combines the pixel-wise segmentation loss, the box loss, and the class loss.

让我们更详细地研究两阶段过程。 对于两阶段网络，您有两个不同的选择。 您可以拥有一个在ROI上工作的联合分支，然后在稍后阶段拆分为蒙版的分割以及类和边界框的预测，或者您可以尽早拆分。 然后，您在两个单独的网络中运行该文件，在这两个版本中，您都有这种多任务丢失，并且将逐像素分段丢失，框丢失和类丢失合并在一起。

Let’s have a look at some examples. These are results again from mask RCNN. You can see that to be honest these are quite impressive results. So, there are really difficult cases. You identify where the persons are and you also see that the different persons, of course, are different instances. So, very impressive results!

让我们看一些例子。 这些也是掩码RCNN的结果。 您可以看到，说实话，这些都是令人印象深刻的结果。 因此，确实有困难的情况。 您确定人员所在的位置，并且还可以看到不同的人员当然是不同的实例。 因此，非常令人印象深刻的结果！

So let’s summarize what we’ve seen so far. The segmentation is commonly solved by architectures analyzing the image and subsequently refining the coarse results. Fully convolutional networks preserve the spatial layout and enable arbitrary input sizes with pooling.

因此，让我们总结一下到目前为止所看到的。 通常通过架构分析图像并随后完善粗略结果来解决分割问题。 完全卷积网络保留空间布局，并通过池化实现任意输入大小。

We can use object detectors and implement them as a sequence of region proposals and classification. Then this leads essentially to the family of RCNN-type of networks. Alternatively, you can go to single-shot detectors. We looked at YOLO which is a very common and very fast technique such as YOLO9000. We looked into RetinaNet if you really have a scale dependency and you want to detect on many different scales like for the example of histological slice processing. So, object detection and segmentation are closely related and combinations are common as you have seen here for the purpose of instance segmentation.

我们可以使用对象检测器并将其实现为区域建议和分类的序列。 然后，这实质上导致了RCNN型网络家族。 或者，您可以转到单次检测器。 我们介绍了YOLO，这是一种非常常见且非常快速的技术，例如YOLO9000。 如果您确实具有比例依赖性，并且想要在许多不同的尺度上进行检测，例如组织学切片处理示例，我们就研究了RetinaNet。 因此，对象检测和分段密切相关，并且组合是常见的，如您在此处为实例分段的目的所见。

Let’s look at what we still have to talk about in this lecture. Coming up very soon are methods to relieve the burden of labeling. So, we will talk about weak annotation. How we can generate labels? This then also leads to the concept of self-supervision which is a very popular topic right now. It’s been very heavily used in order to generate better networks. The methods are able to reuse also sparsely or even completely unlabeled data. We will look into some of the more advanced methods. One idea that I want to show to you later is the use of known operators. How we can integrate knowledge into networks? Which properties does this have? and we also demonstrate some ideas on how we could potentially make parts of networks reusable. So, there are exciting things still coming up.

让我们看看在本讲座中我们仍然要谈论的内容。 很快就会出现减轻标签负担的方法。 因此，我们将讨论弱注释。 我们如何生成标签？ 然后，这也导致了自我监督的概念，这是当前非常流行的话题。 为了生成更好的网络，它已被大量使用。 这些方法还可以稀疏地使用，甚至可以完全重用未标记的数据。 我们将研究一些更高级的方法。 我想稍后向您展示的一个想法是使用已知的运算符。 我们如何将知识整合到网络中？ 有哪些属性？ 我们还将展示一些有关如何使网络的某些部分可重用的想法。 因此，仍然有令人兴奋的事情发生。

I have some comprehensive questions for you like “What is the difference between semantic and instance segmentation?”, “What is the connection to object detection?”, “How can we construct a network which accepts arbitrary input sizes?”, “What is ROI pooling?”, “How can we perform backpropagation through an ROI pooling layer?”, “What are typical measures for the evaluation of segmentations?”, or for example I could ask you to explain a method for instance segmentation.

我对您有一些综合性的问题，例如“语义和实例分割之间的区别是什么？”，“与对象检测的联系是什么？”，“我们如何构建可以接受任意输入大小的网络？”，“什么是ROI池？”，“我们如何通过ROI池层进行反向传播？”，“评估细分的典型方法是什么？”，或者例如，我可以要求您解释实例细分的方法。

I have a couple of further readings in terms of links. So, there is this awesome website by Joseph Redmond the creator of Yolo. I think this is a really nice library that is called darknet. You can also study Joseph’s Redmon’s CV I have the link here. I think if you follow this kind of layout, this will definitely jumpstart your career. Please take your time and also look at the references below. we selected really very good state-of-the-art papers and we can definitely recommend having a look at them. So thank you very much for listening to this lecture and I hope you liked our short excursion to the more applied fields like segmentation and object detection. I hope that this turns out to be useful for you and I also hope that we will see again in one of our next videos. So, thank you very much and goodbye!

关于链接，我还有一些进一步的读物。 因此，Yolo的创建者Joseph Redmond提供了一个很棒的网站。 我认为这是一个非常不错的库，称为darknet 。 您也可以研究Joseph的Redmon的简历，我在这里具有链接 。 我认为，如果您遵循这种布局，那肯定会Swift启动您的职业生涯。 请花一些时间，并查看下面的参考资料。 我们选择了非常好的最新技术论文，我们绝对可以建议您看一看。 因此，非常感谢您收听本讲座，希望您喜欢我们对更广泛应用的领域(如分段和对象检测)的短暂访问。 我希望这对您有用，也希望我们在下一个视频中再次看到。 因此，非常感谢，再见！

If you liked this post, you can find more essays here, more educational material on Machine Learning here, or have a look at our Deep LearningLecture. I would also appreciate a follow on YouTube, Twitter, Facebook, or LinkedIn in case you want to be informed about more essays, videos, and research in the future. This article is released under the Creative Commons 4.0 Attribution License and can be reprinted and modified if referenced. If you are interested in generating transcripts from video lectures try AutoBlog.

如果你喜欢这篇文章，你可以找到这里更多的文章 ，更多的教育材料，机器学习在这里 ，或看看我们的深入 学习 讲座 。 如果您希望将来了解更多文章，视频和研究信息，也欢迎关注YouTube ， Twitter ， Facebook或LinkedIn 。 本文是根据知识共享4.0署名许可发布的 ，如果引用，可以重新打印和修改。 如果您对从视频讲座中生成成绩单感兴趣，请尝试使用AutoBlog 。