Convolution

把各个*input feature maps分别经过一个kernel的卷积结果相加，得到一个* output feature map：

Output feature map’s shape

Convolution layer

Convolution layer的output feature map的shape与下列变量有关：
- input feature map shape iii
- kernel size k" role="presentation" style="position: relative;">kkk (usually odd)
- zero padding ppp
- stride s" role="presentation" style="position: relative;">sss

Name	kkk	p" role="presentation" style="position: relative;">ppp	sss	o" role="presentation" style="position: relative;">ooo
no zero padding & unit stride		000	1" role="presentation" style="position: relative;">111	(i−k)+1(i−k)+1(i − k) + 1
unit strides			111	(i−k)+2p+1" role="presentation" style="position: relative;">(i−k)+2p+1(i−k)+2p+1(i − k) + 2p + 1
Half (or same) paddings	2n+12n+12n+1	⌊k/2⌋=n⌊k/2⌋=n\lfloor k/2\rfloor = n	111	i" role="presentation" style="position: relative;">ii i
Full padding		k−1k−1k-1	111	(i+k)−1" role="presentation" style="position: relative;">(i+k)−1(i+k)−1(i + k) - 1

举例来说，下图表现了Half padding:

Fully-connected layers

Output shape与input shape独立。

注：更多详细的数学运算上的关系可以看这篇paper：A guide to convolution arithmetic for deep learning.

Blocks & Deep neural network

Deep eural Netword由于其deep的特性具有很强的特征提取能力。整个Network的主要部分可以由多个相同的blocks依次串联而成。

以下图中的MemNet为例，red dashed box中的部分就是由基本的Memory Blocks串联成的：

下面提到的各种blocks有一个很关键的特征： they create paths (skip connections) from early layers to later layers.

Residual block & skip connection

Residual block中使用一个skip conncetion，将一个conv layer的input连接到output，即增加了一个identical mapping。

这要做的好处有：
- easier to learn the expected mapping
- alleviate the gradient vanishing problem

Dense Block

在一个dense block的内部，每一对*layers之间都有从前向后*的连接。
也就是说，若一个block内有mmm个layers，则连接总数为m(m−1)2" role="presentation" style="position: relative;">m(m−1)2m(m−1)2\frac{m(m-1)}{2}。

growth rate: 每个layer产生的feature map数量叫做growth rate。
设这个数字为kkk，则一个block的内部的第l" role="presentation" style="position: relative;">lll个layer的输入feature map数量为k0+k×(l−1)k0+k×(l−1)k_0 + k \times (l-1)，其中k0k0k_0为整个block接受的输入数量。

这样做的好处有：
- alleviate the gradient vanishing problem
- strengthen feature propagation and encourage feature reuse: as info about the input or gradient passes through many layers, it can vanish and “wash out” by the time it reaches the end (or beginning) of the network
- reduce the number of parameters: because there is no need to relearn redundant feature-maps, DenseNet layers are very narrow
- regularing effect: it reduces overfitting on tasks with smaller training set sizes

Resiudual Dense Block

Review of Residual Block and Dense Block

Resiudual Dense Block

首先对前面进行了综合，即：
ResDenseBlock←ResBLock+Dense BlockResDenseBlock←ResBLock+Dense Block\text{ResDenseBlock} \leftarrow \text{ResBLock} + \text{Dense Block}

之后加了1×11×11 \times 1的Conv，用于feature fusion。

Residual Dense Network

网络architecture如下，该网络实现SISR (single image super resolution)任务。

其中：
- LR为输入的low resolution图像
- 前两层conv为shallow feature extraction net (SEFNEt)，用于提取浅层信息
- 中间为residuaal dense blocks和dense feature fusion (DFF)
- 最后是用于放大的up-sampling net (UPNet)

从卷积到residual dense network: Convolution, ResNet, DenseNet and ResDenseNet解读相关推荐

超分算法RDN：Residual Dense Network for Image Super-Resolution 超分辨率图像重建
这篇文章总结分析了ResNet 和DenseNet的优缺点,并将其结合,提出了新的结构ResidualDenseNet.文章中对ResNet 和DenseNet以及MemNet都进行了简单的对比分析. ...
【图像复原】RDN论文详解（Residual Dense Network for Image Restoration）
这是CVPR2018的一篇文章,提出了针对图像复原任务的CNN模型RDN(residual dense network). RDN主要是提出了网络结构RDB(residual dense blocks ...
Residual Dense Network for Image Super-Resolution 代码详解
Residual Dense Network for Image Super-Resolution 以下是 RND论文Tensorflow版本实现的源码解析,我假设你已经了解Python的基本语法,和 ...
【文章阅读】【超解像】--Residual Dense Network for Image Super-Resolution
[文章阅读][超解像]–Residual Dense Network for Image Super-Resolution 论文链接:https://arxiv.org/pdf/1802.08797. ...
ResNet（Residual Neural Network）的原理和tf实现
文章目录 1.Highway Network简介 2.模型加深存在的问题 3.为什么深度模型难以训练 3.1.链式法则与梯度弥散 4.ResNet的特点 5.VGGNet-19 VS ResNet-3 ...
图卷积网络GCN（Graph Convolution Network）（一）研究背景和空域卷积
文章首发于个人站点: 图卷积网络GCN(Graph Convolution Network)(一)研究背景和空域图卷积 | 隐舍公众号:[DreamHub] 由于文章篇幅较长,因此将其分解为三部分: ...
【RedNet2018】RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation
RedNet: Residual Encoder-Decoder Network for indoor RGB-D Semantic Segmentation RedNet: 用于室内RGB-D语义分 ...
【读论文】RFN-Nest: An end-to-end residual fusion network for infrared and visible images
[读论文]RFN-Nest: An end-to-end residual fusion network for infrared and visible images 介绍关键词简单介绍网络结 ...
论文翻译：Residual Attention Network for Image Classification
摘要在这项工作中,我们提出了"残余注意力网络",这是一种使用注意力机制的卷积神经网络,可以以端到端的训练方式与最新的前馈网络体系结构结合.我们的剩余注意力网络是通过堆叠注意力模块 ...
[论文阅读] Facial Expression Recognition Using Residual Masking Network
Facial Expression Recognition Using Residual Masking Network 论文链接:https://ieeexplore.ieee.org/docume ...

从卷积到residual dense network: Convolution, ResNet, DenseNet and ResDenseNet解读

Convolution

Output feature map’s shape

Convolution layer

Fully-connected layers

Blocks & Deep neural network

Residual block & skip connection

Dense Block

Resiudual Dense Block

Review of Residual Block and Dense Block

Resiudual Dense Block

Residual Dense Network

从卷积到residual dense network: Convolution, ResNet, DenseNet and ResDenseNet解读相关推荐

最新文章

热门文章