Why we need activation function？

整理自吴恩达深度学习课程 https://mooc.study.163.com/learn/2001281002?tid=2001392029#/learn/content?type=detail&id=2001702018&cid=2001694026

Why we need activation function ？

Deep networks with many many layers, many many hidden layers and turns out that if you use a linear activation function or alternatively if you don’t have an activation function then no matter how many layers your network has, always doing is computing a linear activation function, so you might as well not have any hidden layers.

If you use the linear function here and sigmoid function here, then this model is no more expressive than standard logistic regression without any hidden layer.

The take-home is that a linear hidden layer is more or less useless,because the composition of
two linear function is itself a linear function. So unless you throw a non-linearty in there, then
you are not computing more interesting functions even as you good deeper in the network

I, blogger of this post, think that if you are suing linear function, you deep neural network will not get the higher level features we expect.

The hidden units should not use the linear activation functions, they could use ReLU or tanh or leaky ReLU or maybe somethings else. So the one place you might use as linear activation function is usually the output layer. But other that, using a linear activation function in a hidden layer except for some very special circumstances relating to compression. Using linear activation is extremely rare.

Why we need activation function？相关推荐

激活函数 activation function
文章目录激活函数 activation function Sigmoid Sigmoid 反向传播 Tanh ReLU Dead ReLU Problem 产生的原因激活函数 activation ...
Homepage Machine Learning Algorithm 浅谈深度学习中的激活函数 - The Activation Function in Deep Learning
原文地址:http://www.cnblogs.com/rgvb178/p/6055213.html 版权声明:本文为博主原创文章,未经博主允许不得转载. 激活函数的作用首先,激活函数不是真的要去激 ...
激活函数（Activation Function）及十大常见激活函数
目录 1 激活函数的概念和作用 1.1 激活函数的概念 1.2 激活函数的作用 1.3 通俗地理解一下激活函数(图文结合) 1.3.1 无激活函数的神经网络 1.3.2 带激活函数的神经网络 2 神经 ...
深度学习中常见的10种激活函数（Activation Function）总结
目录一:简介二:为什么要用激活函数三:激活函数的分类四:常见的几种激活函数 4.1.Sigmoid函数 4.2.Tanh函数 4.3.ReLU函数 4.4.Leaky Relu函数 4.5.P ...
第3.1章卷积神经网络(CNN)——Conv、Pool、FC、Activation Function、BN各个层的作用及原理
第3.1章卷积神经网络CNN-不同层的作用一.Convolution(CONV) 二.Pooling(POOL) 三.Fully Connected(FC) 四.Activation Functi ...
浅谈深度学习中的激活函数 - The Activation Function in Deep Learning
原文地址:http://www.cnblogs.com/rgvb178/p/6055213.html 版权声明:本文为博主原创文章,未经博主允许不得转载. 激活函数的作用首先,激活函数不是真的要去激 ...
【机器学习】神经网络-激活函数-面面观(Activation Function)
神经网络之激活函数(Activation Function) 本博客仅为作者记录笔记之用,不免有很多细节不对之处. 还望各位看官能够见谅,欢迎批评指正. 更多相关博客请猛戳:http://blog.c ...
通俗理解神经网络之激励函数(Activation Function)
本文整理自博客.知乎上关于激励函数的博客和回答.部分图片引用自知乎,如有侵权,请联系作者. 关于神经网络激励函数的作用,常听到的解释是:不使用激励函数的话,神经网络的每层都只是做线性变换,多层输入叠加 ...
聊一聊深度学习的activation function
转载自:https://zhuanlan.zhihu.com/p/25110450 TLDR (or the take-away) 优先使用ReLU (Rectified Linear Unit) 函 ...

Why we need activation function？

Why we need activation function ？

Why we need activation function？相关推荐

最新文章

热门文章