深度学习深度前馈网络

FAU深度学习讲义 (FAU Lecture Notes in Deep Learning)

These are the lecture notes for FAU’s YouTube Lecture “Deep Learning”. This is a full transcript of the lecture video & matching slides. We hope, you enjoy this as much as the videos. Of course, this transcript was created with deep learning techniques largely automatically and only minor manual modifications were performed. If you spot mistakes, please let us know!

这些是FAU YouTube讲座“ 深度学习 ”的讲义。 这是演讲视频和匹配幻灯片的完整记录。 我们希望您喜欢这些视频。 当然，此成绩单是使用深度学习技术自动创建的，并且仅进行了较小的手动修改。 如果发现错误，请告诉我们！

导航 (Navigation)

Previous Lecture / Watch this Video / Top Level / Next Lecture

上一个讲座 / 观看此视频 / 顶级 / 下一个讲座

CC BY 4.0 from the 深度学习讲座中 Deep Learning Lecture. CC BY 4.0下的图像。

Welcome everybody to our lecture on deep learning! Today, we want to go into the topic. We want to introduce some of the important concepts and theories that have been fundamental to the field. Today’s topic will be feed-forward networks and feed-forward networks are essentially the main configuration of neural networks as we use them today. So in the next couple of videos, we want to talk about the first models and some ideas behind them. We also introduce a bit of theory. One important block will be about Universal function approximation where we will essentially show that neural networks are able to approximate any kind of function. This will then be followed by the introduction of the softmax function and some activations. In the end, we want to talk a bit about how to optimize such parameters and in particular, we will talk about the backpropagation algorithm.

欢迎大家参加我们的深度学习讲座！今天，我们想进入主题。我们想介绍一些对该领域至关重要的重要概念和理论。今天的主题将是前馈网络，而前馈网络本质上是我们今天使用它们时神经网络的主要配置。因此，在接下来的两个视频中，我们想讨论第一个模型及其背后的一些想法。我们还将介绍一些理论。一个重要的块将是关于通用函数逼近的，我们将从本质上证明神经网络能够逼近任何一种函数。然后将引入softmax函数和一些激活。最后，我们想谈谈如何优化这些参数，尤其是，我们将讨论反向传播算法。

So let’s start with the model and what you heard already is the perceptron. We already talked about this which was essentially a function that would map any high dimensional input to an inner product of the weight vector and the input. Then, we are only interested in the signed distance that is computed. You can interpret this essentially as you see above on the right-hand side. The decision boundary is shown in red and what you’re computing with the inner product is essentially a signed distance of a new sample to this decision boundary. If we consider only the sign, we can decide whether we are on one side or the other.

因此，让我们从模型开始，您已经听到的是感知器。我们已经讨论过这个了，它本质上是一个将任何高维输入映射到权重向量和输入的内积的函数。然后，我们只对计算出的有符号距离感兴趣。您可以按照上面右侧的方法从本质上解释这一点。决策边界以红色显示，您对内积的计算本质上是新样本到该决策边界的有符号距离。如果仅考虑标志，则可以决定我们在一侧还是另一侧。

Now, if you look at classical pattern recognition and machine learning, we would still follow a so-called pattern recognition pipeline. We have some measurement that is converted and pre-processed in order to increase the quality, e.g. decrease noise. In the pre-processing, we essentially stay in the same domain as the input. So if you have an image as input, the output of the pre-processing will also be an image, but with probably better properties towards the classification task. Then, we want to do feature extraction. You remember the example with the apples and pears. From these, we extract features which then result in some high dimensional vector space. We can then go ahead and do the classification.

现在，如果您研究经典模式识别和机器学习，我们仍然会遵循所谓的模式识别管道。我们对一些测量值进行了转换和预处理，以提高质量，例如降低噪声。在预处理中，我们基本上与输入保持在相同的域中。因此，如果您有图像作为输入，则预处理的输出也将是图像，但对于分类任务可能具有更好的属性。然后，我们要进行特征提取。您还记得苹果和梨的例子。从这些中，我们提取特征，然后产生一些高维向量空间。然后，我们可以继续进行分类。

Now, what we’ve seen in the perceptron is that we are able to model linear decision boundaries. This immediately then led to the observation that perceptrons cannot solve the logical exclusive or — the so-called XOR. You can see the visualization of the XOR problem above on the left-hand side. So, imagine you have some kind of distribution of classes where the top left and the bottom right is blue and the other class is bottom left and top right. This is inspired by the logical XOR function. You will not be able to separate those two point clouds with a single linear decision boundary. So, you either need curves or you use multiple lines. With a single perceptron, you will not be able to solve this problem. Because people have been arguing: “Look we can model logical functions with perceptrons. If we build perceptrons on perceptrons, we can essentially build all of the logic!”

现在，我们在感知器中看到的是我们能够对线性决策边界建模。然后，这立即导致观察到感知器无法解决逻辑异或或所谓的XOR。您可以在上方的左侧看到XOR问题的可视化。因此，假设您有某种类的分布，其中左上角和右下角是蓝色，另一个类是左下角和右上角。这是受逻辑XOR函数启发的。您将无法使用单个线性决策边界来分离这两个点云。因此，您需要曲线或使用多条线。使用单个感知器，您将无法解决此问题。因为人们一直在争论：“看起来我们可以用感知器对逻辑函数建模。如果我们在感知器上构建感知器，那么我们基本上可以构建所有逻辑！”

Now, if you can’t build XOR, then you’re probably not able to describe the entire logic and therefore, we will never achieve strong AI. This was a period of time when all funding to artificial intelligence research was tremendously cut down and people would not get any new grants. They would not get money to support the research. Hence, this period became known as the “AI Winter”.

现在，如果您无法构建XOR，那么您可能无法描述整个逻辑，因此，我们将永远无法实现强大的AI。在这段时间里，用于人工智能研究的所有资金都被大大削减了，人们将不会获得任何新的资助。他们没有钱来支持这项研究。因此，这一时期被称为“人工智能冬季”。

Things changed with the introduction of the multi-layer perceptron. This is now the expansion of the perceptron. You do not just do a single neuron, but you use multiple of those neurons and you arrange them in layers. So here you can see a very simple draft. So, it is very similar to the perceptron. You have essentially some inputs and some weights. Now, you can see that it’s not just a single sum, but we have several of those sums that go through a non-linearity. Then, they assign weights again and summarize again to go into another non-linearity.

随着多层感知器的引入，情况发生了变化。现在这是感知器的扩展。您不仅要处理单个神经元，还要使用多个神经元，然后将它们分层排列。因此，在这里您可以看到一个非常简单的草稿。因此，它与感知器非常相似。您实际上具有一些输入和权重。现在，您可以看到它不仅是一个总和，而且其中有几个总和都是非线性的。然后，他们再次分配权重并再次汇总，以得出另一个非线性。

This is very interesting because we can use multiple neurons. We can now also model nonlinear decision boundaries. You can go on and then arrange this in layers. So what you typically do is, you have some input layer. This is our vector x. Then, you have several perceptrons that you arrange in hidden layers. They’re called hidden because they do not immediately observe the input. They assign weights, then compute something, and only at the very end, at the output, you have a layer again where you can observe what’s actually happening. All of these weights that are in between in those hidden layers, they are not directly observable. Here, you only observe them when you put some input in, compute the activations, and then at the very end, you can obtain the output. So, this is where you can actually observe what’s happening in your system.

这非常有趣，因为我们可以使用多个神经元。现在，我们还可以对非线性决策边界建模。您可以继续，然后按层次排列。因此，通常要做的是，有一些输入层。这是我们的向量x。然后，您将有几个感知器布置在隐藏层中。之所以称它们为“隐藏”，是因为它们没有立即观察到输入。他们分配权重，然后计算一些东西，直到最后，在输出端，您又有了一个图层，可以在其中观察实际发生的情况。这些隐藏层之间的所有权重都是无法直接观察到的。在这里，仅当您输入一些输入，计算激活量并最终获得输出时，您才能观察到它们。因此，您可以在这里实际观察系统中发生的情况。

x) on a compact set. Image under x )。 CC BY 4.0 from the 深度学习讲座中 Deep Learning Lecture. CC BY 4.0下的图像。

Now, we will look into the so-called universal function approximator. This is actually just a network with a single hidden layer. Universal function approximation is a fundamental piece of theory because it tells us that with a single hidden layer, we can approximate any continuous function. So, let’s look a bit into this theorem. It starts as a formal definition. We have some

深度学习深度前馈网络_深度学习前馈网络中的讲义第1部分相关推荐

深度学习深度前馈网络_深度学习前馈网络中的讲义第4部分
深度学习深度前馈网络 FAU深度学习讲义 (FAU Lecture Notes in Deep Learning) These are the lecture notes for FAU's YouT ...
重拾强化学习的核心概念_强化学习的核心概念
重拾强化学习的核心概念 By Hannah Peterson and George Williams (gwilliams@gsitechnology.com) 汉娜·彼得森 ( Hannah Pet ...
前馈神经网络_深度学习基础理解：以前馈神经网络为例
区别于传统统计机器学习的各类算法,我们从本篇开始探索深度学习模型.深度学习在应用上的重要性现如今已毋庸置疑,从2012年燃爆ImageNet,到2016年的AlphaGo战胜李世石,再到2018年的B ...
python实现胶囊网络_深度学习精要之CapsuleNets理论与实践（附Python代码）
摘要: 本文对胶囊网络进行了非技术性的简要概括,分析了其两个重要属性,之后针对MNIST手写体数据集上验证多层感知机.卷积神经网络以及胶囊网络的性能. 神经网络于上世纪50年代提出,直到最近十年里才得 ...
深度学习之对象检测_深度学习时代您应该阅读的12篇文章，以了解对象检测
深度学习之对象检测前言 (Foreword) As the second article in the "Papers You Should Read" series, we a ...
深度学习领域专业词汇_深度学习时代的人文领域专业知识
深度学习领域专业词汇 It's a bit of an understatement to say that Deep Learning has recently become a hot topic ...
深度学习背后的数学_深度学习背后的简单数学
深度学习背后的数学 Deep learning is one of the most important pillars in machine learning models. It is based ...
强化学习之基础入门_强化学习基础
强化学习之基础入门 Reinforcement learning is probably one of the most relatable scientific approaches that re ...
半学期学计算机有感论文,【计算机学习心得论文】_计算机学习心得论文参考资料-毕业论文范文网...
英语学习的一点心得英语学习的一点心得英语学习的一点心得,一提到学习英语,很多同学就觉得是个头疼的问题.更有同学说,我天生没有英语细胞.我觉得,英语成绩上不去,还是跟自己的学习态度和方法有很大关系.英 ...

深度学习深度前馈网络_深度学习前馈网络中的讲义第1部分

FAU深度学习讲义 (FAU Lecture Notes in Deep Learning)

导航 (Navigation)

深度学习深度前馈网络_深度学习前馈网络中的讲义第1部分相关推荐

最新文章

热门文章