FAU深度学习讲义 (FAU Lecture Notes in Deep Learning)

These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. If you spot mistakes, please let us know!

这些是FAU YouTube讲座“ 深度学习 ”的 讲义 。 这是演讲视频和匹配幻灯片的完整记录。 我们希望您喜欢这些视频。 当然，此成绩单是使用深度学习技术自动创建的，并且仅进行了较小的手动修改。 如果发现错误，请告诉我们！

导航 (Navigation)

Previous Lecture / Watch this Video / Top Level / Next Lecture

上一个讲座 / 观看此视频 / 顶级 / 下一个讲座

Thanks for tuning in to the next video of deep learning. So, what I want to show you in this video are a couple of limitations of deep learning. So, you may wonder are there any limitations? Are we done yet? Aren’t we learning something here that will solve all of the problems?

感谢您收看下一个深度学习视频。 因此，我想在此视频中向您展示的是深度学习的两个限制。 因此，您可能想知道是否存在任何限制？ 我们完成了吗？ 我们不是在这里学到可以解决所有问题的东西吗？

Well of course there are some limitations. For example tasks like image captioning yield impressive results. You can see that the networks are able to identify the baseball player, the girl in a pink dress jumping in the air, or even people playing the guitar.

好吧，当然有一些限制。 例如，图像字幕之类的任务会产生令人印象深刻的结果。 您会看到，网络能够识别棒球运动员，身穿粉红色连衣裙的女孩在空中跳跃，甚至是在弹吉他的人。

So let’s look at some errors. Here on the left, you can see, this is clearly not a baseball bat. Also, this isn’t a cat in the center image and they are also slight errors like the one on the right-hand side: The cat on top of the suitcases isn’t black.

因此，让我们看一些错误。 在左边，您可以看到，这显然不是棒球棍。 另外，这不是中心图中的猫，并且它们也像右侧的猫一​​样有轻微的错误：手提箱顶部的猫不是黑色的。

Sometimes they’re even plain errors like here in the left image: There, I don’t see a horse in the middle of the road and also on the right image there is no woman holding a teddy bear in front of a mirror.

有时它们甚至是简单的错误，如左图所示：在那儿，我看不到马在中间，在右图上也没有女人在镜子前抱着泰迪熊。

So, the reason for this is of course there’s a couple of challenges and one major challenge is training data. Deep learning applications require huge manually annotated data sets and these are hard to obtain. Annotation is time-consuming and expensive and often ambiguous. So, as you’ve seen already in the image net challenge, sometimes it’s not clear which label to assign, and obviously you would have to assign a distribution of labels. Also, we see that even in the human annotations, there are typical errors. What you have to do in order to get a really good representation of the labels, you actually have to ask two or even more experts to do the entire labeling process. Then, you can find out the instances where you have a very sharp distribution of labels. These are typical prototypes and broad distributions of labels are images where people are not sure. If we have such problems, then we typically get a significant drop in performance. So the question is how far can we get simulations for example to expand training data.

因此，这样做的原因当然是存在两个挑战，其中一个主要挑战是训练数据。 深度学习应用程序需要庞大的手动注释数据集，而这些数据集很难获得。 注释既费时又昂贵，而且常常含糊不清。 因此，正如您在图像网络挑战中已经看到的那样，有时不清楚要分配哪个标签，显然您必须分配标签的分布。 此外，我们看到即使在人工注释中，也存在典型错误。 为了获得标签的真正良好表现，您实际上必须要请两个或更多的专家来完成整个标签过程。 然后，您可以找出标签分布非常清晰的实例。 这些是典型的原型，标签的广泛分布是人们不确定的图像。 如果遇到此类问题，通常会导致性能大幅下降。 所以问题是，例如，我们可以得到多远的模拟以扩展训练数据。

Of course, there are also challenges with trust and reliability. So, verification is mandatory for high-risk applications, and regulators can be very strict about those. They really want to understand what’s happening in those high-risk systems. End-to-end learning essentially prohibits to identify how the individual parts work. So, it’s very hard for regulators to tell what part does what and why the system actually works. We must admit at this point that this is largely unsolved to a large degree. It’s difficult to tell which part of the network is doing what. Modular approaches that are based on classical algorithms may be one approach to solve these problems in the future.

当然，信任和可靠性也面临挑战。 因此，对于高风险应用程序，验证是必不可少的，而监管机构对此可能非常严格。 他们真的很想了解那些高风险系统中正在发生的事情。 端到端学习本质上禁止识别各个部分的工作方式。 因此，监管者很难说出哪个部分在做什么以及为什么该系统真正起作用。 在这一点上，我们必须承认，这在很大程度上尚未解决。 很难说出网络的哪个部分在做什么。 基于经典算法的模块化方法可能是将来解决这些问题的一种方法。

This brings us to the future directions and something that we like to do here in Erlangen is learning algorithms. So for example, you can look at the classical computer tomography which is expressed in the filtered back-projection formula. You have to filter along the projection direction and then a summation over the angle in order to produce a final image. So this convolution and back-projection then can actually be expressed in terms of linear operators. As such, they are essentially matrix multiplications.

这将我们带到了未来的方向，我们在Erlangen中想要做的事情是学习算法。 因此，例如，您可以查看经典的计算机断层扫描技术，该技术以过滤后的反投影公式表示。 您必须沿投影方向进行过滤，然后对角度进行求和才能生成最终图像。 因此，这种卷积和反投影实际上可以用线性算子表示。 因此，它们本质上是矩阵乘法。

Now, those matrix multiplications can be implemented as a neural network and you essentially then have an algorithm or a network design that can be trained for specific purposes.

现在，可以将这些矩阵乘法实现为神经网络，然后从本质上讲，您便有了可以针对特定目的进行训练的算法或网络设计。

So here, we extend this approach in order to apply it to fan-beam projection data. This is a slight modification of the algorithm. There are cases that cannot be solved like the limited angle situation.

因此，在这里，我们扩展了此方法，以便将其应用于扇形束投影数据。 这是对该算法的略微修改。 有些情况无法解决，例如有限的角度情况。

In this image, you see a full scan and this produces a reasonable CT image. However, if you’re missing only twenty degrees of rotation you already see severe artifacts:

在此图像中，您将看到完整扫描，并生成了合理的CT图像。 但是，如果仅缺少二十度旋转，则您已经看到了严重的伪影：

Now, if you take the idea of converting your reconstruction algorithm into a neural network and retrain it on some training data. Here it’s only 15 images and you can see that even on unseen data, we are able to recover most of the lost information:

现在，如果您考虑将重构算法转换为神经网络并在一些训练数据上对其进行重新训练的想法。 这里只有15张图片，您可以看到，即使是在看不见的数据上，我们也可以恢复丢失的大部分信息：

Now, if you look at the top part of this image you can see that there is a reduction of mass. We show line plots following the red line in the left image on the right-hand side:

现在，如果您查看此图像的顶部，可以看到质量有所减少。 我们在左侧图像中的红线后面显示线图：

Now here in green, you can see the reference image that is largely unaffected. As soon as you introduce the angle limitation, you end up with the red curve which shows these artifacts in the top part of the image. Now, if you further go ahead and take our deep learning method, you end up with the blue curve which largely reduces problems that have been introduced by the angular and limitation. Now, the fun part about this is because our algorithm has been inspired by a traditional CT reconstruction algorithm, all of those layers have interpretations: They are linked to a specific function. What do you typically do for such a short scan, is that you weight down rays that have been measured twice, such that the opposing rays exactly sum to one.

现在，这里为绿色，您可以看到基本不受影响的参考图像。 引入角度限制后，您最终会得到红色曲线，该红色曲线在图像的顶部显示了这些伪像。 现在，如果您继续前进并采用我们的深度学习方法，您将得到一条蓝色曲线，该曲线大大减少了角度和限制所带来的问题。 现在，有趣的部分是因为我们的算法是受传统CT重建算法启发的，所有这些层都具有以下解释：它们与特定功能链接。 对于这么短的扫描，您通常会做什么，就是权衡两次测量的光线，以使相对的光线恰好合计为一。

You can see that here the Parker Weights in this figure. Now, if we train our algorithm, the main changes are the Parker weights. What happens is that we can see an increase of weight in particular in rays that run through the area that has the angular limitation after training:

您可以在此图中看到派克砝码。 现在，如果我们训练算法，则主要的变化是帕克权重。 发生的是，我们可以看到重量增加，特别是在训练后穿过具有角度限制的区域的光线中：

So, the network learns to use the information from a slightly different direction in those rays that have not been measured. You can even go ahead and then combine this reconstruction method with additional de-streaking and denoising steps and as we will show towards the end of this lecture. As a result, we can dramatically improve also in the low contrast information.

因此，网络学会从尚未被测量的射线中以稍微不同的方向使用信息。 您甚至可以继续，然后将此重构方法与其他去条纹和去噪步骤相结合，正如我们将在本讲座结束时所示。 结果，我们还可以显着改善低对比度信息。

Here, you see an image of the full scan reference on the top left, the neural network output on the top right that still has significant streaks, and on the bottom right, you see a streak reduction network. It is able to really reduce those streaks that have been caused by the angular limitation. Compared to just a denoising approach on the bottom left, you can see that those streaks would be diminished but they still would be present. Only such a trained method that understands what the problem is, is actually able to reduce those artifacts efficiently.

在这里，您会在左上角看到完整扫描参考的图像，在右上角仍然有明显条纹的神经网络输出，在右下角会看到条纹减少网络。 它确实能够减少由角度限制引起的条纹。 与仅左下角的降噪方法相比，您可以看到这些条纹会减少，但仍然存在。 实际上，只有这样一种了解问题所在的经过训练的方法才能有效地减少那些伪影。

So next time in deep learning, we want to look at basic pattern recognition and machine learning. Which basic rules are there and how this is in relation to deep learning? Then, we want to go ahead and talk about the perceptron which is the basic unit of a neural network. From those perceptrons, you can build those really deep networks that we have been featuring in this and previous videos. So, I hope you like this video and it would be really great if you tune in next time. Thank you!

因此，下次在深度学习中，我们想看一下基本模式识别和机器学习。 有哪些基本规则？这与深度学习有何关系？ 然后，我们要继续讨论感知器，它是神经网络的基本单元。 从这些感知器中，您可以构建在本视频和以前的视频中我们已经介绍过的那些真正深入的网络。 因此，我希望您喜欢这个视频，如果下次收看，那真的很棒。 谢谢！

If you liked this post, you can find more essays here, more educational material on Machine Learning here, or have a look at our Deep Learning Lecture. I would also appreciate a clap or a follow on YouTube, Twitter, Facebook, or LinkedIn in case you want to be informed about more essays, videos, and research in the future. This article is released under the Creative Commons 4.0 Attribution License and can be reprinted and modified if referenced.

如果你喜欢这篇文章，你可以找到这里更多的文章 ，更多的教育材料，机器学习在这里 ，或看看我们的深入 学习 讲座 。 如果您想在以后了解更多文章，视频和研究信息，也欢迎在YouTube ， Twitter ， Facebook或LinkedIn上进行拍手或追随。 本文是根据知识共享4.0署名许可发布的 ，如果引用，可以重新打印和修改。