深度学习--Inception-ResNet-v1网络结构
Inception V4的网络结构如下:
从图中可以看出,输入部分与V1到V3的输入部分有较大的差别,这样设计的目的为了:使用并行结构、不对称卷积核结构,可以在保证信息损失足够小的情况下,降低计算量。结构中1*1的卷积核也用来降维,并且也增加了非线性。
Inception-ResNet-v2与Inception-ResNet-v1的结构类似,除了stem部分。Inception-ResNet-v2的stem与V4的结构类似,Inception-ResNet-v2的输出chnnel要高。Reduction-A相同,Inception-ResNet-A、Inception-ResNet-B、Inception-ResNet-C和Reduction-B的结构与v1的类似,只不过输出的channel数量更多。
Inception-ResNet-v1的总体网络结构如下所示:
Inception-ResNet-v1的Stem与V3的结构是一致的。
接下来主要说一下Inception-ResNet-v1的网络结构及代码的实现部分。
Stem结构
stem结构与V3的Stem结构类似。
对应的代码为
with slim.arg_scope([slim.conv2d, slim.max_pool2d, slim.avg_pool2d],stride=1, padding='SAME'):# 149 x 149 x 32net = slim.conv2d(inputs, 32, 3, stride=2, padding='VALID', scope='Conv2d_1a_3x3')end_points['Conv2d_1a_3x3'] = net# 147 x 147 x 32net = slim.conv2d(net, 32, 3, padding='VALID',
scope='Conv2d_2a_3x3')end_points['Conv2d_2a_3x3'] = net# 147 x 147 x 64net = slim.conv2d(net, 64, 3, scope='Conv2d_2b_3x3')
end_points['Conv2d_2b_3x3'] = net# 73 x 73 x 64net = slim.max_pool2d(net, 3, stride=2, padding='VALID', scope='MaxPool_3a_3x3')end_points['MaxPool_3a_3x3'] = net# 73 x 73 x 80net = slim.conv2d(net, 80, 1, padding='VALID',
scope='Conv2d_3b_1x1')end_points['Conv2d_3b_1x1'] = net# 71 x 71 x 192net = slim.conv2d(net, 192, 3, padding='VALID',
scope='Conv2d_4a_3x3')end_points['Conv2d_4a_3x3'] = net# 35 x 35 x 256net = slim.conv2d(net, 256, 3, stride=2, padding='VALID',
scope='Conv2d_4b_3x3')end_points['Conv2d_4b_3x3'] = net
Inception-resnet-A模块
Inception-resnet-A模块是要重复5次的,网络结构为:
对应的代码表示为:
# Inception-Renset-A
def block35(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):"""Builds the 35x35 resnet block."""with tf.variable_scope(scope, 'Block35', [net], reuse=reuse):with tf.variable_scope('Branch_0'):# 35 × 35 × 32tower_conv = slim.conv2d(net, 32, 1, scope='Conv2d_1x1')with tf.variable_scope('Branch_1'):# 35 × 35 × 32tower_conv1_0 = slim.conv2d(net, 32, 1, scope='Conv2d_0a_1x1')# 35 × 35 × 32tower_conv1_1 = slim.conv2d(tower_conv1_0, 32, 3, scope='Conv2d_0b_3x3')with tf.variable_scope('Branch_2'):# 35 × 35 × 32tower_conv2_0 = slim.conv2d(net, 32, 1, scope='Conv2d_0a_1x1')# 35 × 35 × 32tower_conv2_1 = slim.conv2d(tower_conv2_0, 32, 3, scope='Conv2d_0b_3x3')# 35 × 35 × 32tower_conv2_2 = slim.conv2d(tower_conv2_1, 32, 3, scope='Conv2d_0c_3x3')# 35 × 35 × 96mixed = tf.concat([tower_conv, tower_conv1_1, tower_conv2_2], 3)# 35 × 35 × 256up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,activation_fn=None, scope='Conv2d_1x1')# 使用残差网络scale = 0.17net += scale * upif activation_fn:net = activation_fn(net)return net# 5 x Inception-resnet-A
net = slim.repeat(net, 5, block35, scale=0.17)
end_points['Mixed_5a'] = net
Reduction-A结构
Reduction-A中含有4个参数k、l、 m、 n,它们对应的值分别为:192, 192, 256, 384,在该层网络结构,输入为35×35×256,输出为17×17×896.
def reduction_a(net, k, l, m, n):# 192, 192, 256, 384with tf.variable_scope('Branch_0'):# 17×17×384tower_conv = slim.conv2d(net, n, 3, stride=2, padding='VALID',scope='Conv2d_1a_3x3')with tf.variable_scope('Branch_1'):# 35×35×192tower_conv1_0 = slim.conv2d(net, k, 1, scope='Conv2d_0a_1x1')# 35×35×192tower_conv1_1 = slim.conv2d(tower_conv1_0, l, 3,scope='Conv2d_0b_3x3')# 17×17×256tower_conv1_2 = slim.conv2d(tower_conv1_1, m, 3,stride=2, padding='VALID',scope='Conv2d_1a_3x3')with tf.variable_scope('Branch_2'):# 17×17×256tower_pool = slim.max_pool2d(net, 3, stride=2, padding='VALID',scope='MaxPool_1a_3x3')# 17×17×896net = tf.concat([tower_conv, tower_conv1_2, tower_pool], 3)return net# Reduction-A
with tf.variable_scope('Mixed_6a'):net = reduction_a(net, 192, 192, 256, 384)end_points['Mixed_6a'] = net
Inception-Resnet-B
Inception-Resnet-B模块是要重复10次,输入为17×17×896,输出为17×17×896,网络结构为:
# Inception-Renset-B
def block17(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):"""Builds the 17x17 resnet block."""with tf.variable_scope(scope, 'Block17', [net], reuse=reuse):with tf.variable_scope('Branch_0'):# 17*17*128tower_conv = slim.conv2d(net, 128, 1, scope='Conv2d_1x1')with tf.variable_scope('Branch_1'):# 17*17*128tower_conv1_0 = slim.conv2d(net, 128, 1, scope='Conv2d_0a_1x1')# 17*17*128tower_conv1_1 = slim.conv2d(tower_conv1_0, 128, [1, 7],scope='Conv2d_0b_1x7')# 17*17*128tower_conv1_2 = slim.conv2d(tower_conv1_1, 128, [7, 1],scope='Conv2d_0c_7x1')# 17*17*256mixed = tf.concat([tower_conv, tower_conv1_2], 3)# 17*17*896up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,activation_fn=None, scope='Conv2d_1x1')net += scale * upif activation_fn:net = activation_fn(net)return net# 10 x Inception-Resnet-B
net = slim.repeat(net, 10, block17, scale=0.10)
end_points['Mixed_6b'] = net
Reduction-B
Reduction-B的输入为17*17*896,输出为8*8*1792。网络结构为:
对应的代码为:
def reduction_b(net):with tf.variable_scope('Branch_0'):# 17*17*256tower_conv = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')# 8*8*384tower_conv_1 = slim.conv2d(tower_conv, 384, 3, stride=2,padding='VALID', scope='Conv2d_1a_3x3')with tf.variable_scope('Branch_1'):# 17*17*256tower_conv1 = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')# 8*8*256tower_conv1_1 = slim.conv2d(tower_conv1, 256, 3, stride=2,padding='VALID', scope='Conv2d_1a_3x3')with tf.variable_scope('Branch_2'):# 17*17*256tower_conv2 = slim.conv2d(net, 256, 1, scope='Conv2d_0a_1x1')# 17*17*256tower_conv2_1 = slim.conv2d(tower_conv2, 256, 3,scope='Conv2d_0b_3x3')# 8*8*256tower_conv2_2 = slim.conv2d(tower_conv2_1, 256, 3, stride=2,padding='VALID', scope='Conv2d_1a_3x3')with tf.variable_scope('Branch_3'):# 8*8*896tower_pool = slim.max_pool2d(net, 3, stride=2, padding='VALID',scope='MaxPool_1a_3x3')# 8*8*1792net = tf.concat([tower_conv_1, tower_conv1_1,tower_conv2_2, tower_pool], 3)return net
# Reduction-B
with tf.variable_scope('Mixed_7a'):net = reduction_b(net)
end_points['Mixed_7a'] = net
Inception-Resnet-C结构
Inception-Resnet-C结构重复5次。它输入为8*8*1792,输出为8*8*1792。对应的结构为:
对应的代码为:
# Inception-Resnet-C
def block8(net, scale=1.0, activation_fn=tf.nn.relu, scope=None, reuse=None):"""Builds the 8x8 resnet block."""with tf.variable_scope(scope, 'Block8', [net], reuse=reuse):with tf.variable_scope('Branch_0'):# 8*8*192tower_conv = slim.conv2d(net, 192, 1, scope='Conv2d_1x1')with tf.variable_scope('Branch_1'):# 8*8*192tower_conv1_0 = slim.conv2d(net, 192, 1, scope='Conv2d_0a_1x1')# 8*8*192tower_conv1_1 = slim.conv2d(tower_conv1_0, 192, [1, 3],scope='Conv2d_0b_1x3')# 8*8*192tower_conv1_2 = slim.conv2d(tower_conv1_1, 192, [3, 1],scope='Conv2d_0c_3x1')# 8*8*384mixed = tf.concat([tower_conv, tower_conv1_2], 3)# 8*8*1792up = slim.conv2d(mixed, net.get_shape()[3], 1, normalizer_fn=None,activation_fn=None, scope='Conv2d_1x1')# scale=0.20net += scale * upif activation_fn:net = activation_fn(net)return net
# 5 x Inception-Resnet-C
net = slim.repeat(net, 5, block8, scale=0.20)
end_points['Mixed_8a'] = net
但是在facenet中,接下来又是一层Inception-Resnet-C,但是它没有重复,并且没有激活函数。输入与输出大小相同。
net = block8(net, activation_fn=None)
end_points['Mixed_8b'] = net
结果输出
结果输出包含Average Pooling和Dropout (keep 0.8)及Softmax三层,这里我们以facenet中为例:具体的代码如下:
with tf.variable_scope('Logits'):end_points['PrePool'] = net#pylint: disable=no-member# Average Pooling层,输出为8×8×1792net = slim.avg_pool2d(net, net.get_shape()[1:3], padding='VALID',scope='AvgPool_1a_8x8')#扁平除了batch_size维度的其它维度。使输出变为:[batch_size, ...]net = slim.flatten(net)#dropout层net = slim.dropout(net, dropout_keep_prob, is_training=is_training,scope='Dropout')end_points['PreLogitsFlatten'] = net# 全链接层。输出为batch_size×128net = slim.fully_connected(net, bottleneck_layer_size, activation_fn=None,scope='Bottleneck', reuse=False)
至此,inception_resnet_v1网络结构就结束了,但facenet的代码分析未完,待续~~~~
深度学习--Inception-ResNet-v1网络结构相关推荐
- 【深度学习】ResNet系列网络结构
[深度学习]ResNet系列网络结构 ResNet中Residual的功能 DNN的反向传播算法 梯度弥散和梯度爆炸 网络退化 ResNet中Residual的功能 ResNet系列网络结构 结语 R ...
- 【深度学习之ResNet】——深度残差网络—ResNet总结
目录 论文名称:Deep Residual Learning for Image Recognition 摘要: 1.引言 2.为什么会提出ResNet残差网络呢? 3.深度残差网络结构学习(Deep ...
- 深度学习笔记---多尺度网络结构归类总结
目录 1.什么是图像金字塔 1.1 高斯金字塔 ( Gaussian pyramid): 1.2 拉普拉斯金字塔(Laplacian pyramid) 1.3 DOG金字塔 2. 多尺度网络(MTCN ...
- 【深度学习】ResNet——CNN经典网络模型详解(pytorch实现)
建议大家可以实践下,代码都很详细,有不清楚的地方评论区见~ 1.前言 ResNet(Residual Neural Network)由微软研究院的Kaiming He等四名华人提出,通过使用ResNe ...
- 【深度学习】ResNet残差网络 ResidualBlock残差块实现(pytorch) | 跟着李沐学AI笔记 | ResNet18进行猫狗分类
文章目录 前言 一.卷积的相关计算公式(复习) 二.残差块ResidualBlock复现(pytorch) 三.残差网络ResNet18复现(pytorch) 四.直接调用方法 五.具体实践(ResN ...
- 深度学习:ResNet(残差网络)
ResNet在ILSVRC2015竞赛中惊艳亮相,一下子将网络深度提升到152层,将错误率降到了3.57,在图像识别错误率和网络深度方面,比往届比赛有了非常大的提升,ResNet毫无悬念地夺得了ILS ...
- 【深度学习】ResNet网络详解
文章目录 ResNet 参考 结构概况 conv1与池化层 残差结构 Batch Normalization 总结 ResNet 参考 ResNet论文: https://arxiv.org/abs/ ...
- 深度学习—常见的神经网络结构
一.卷积神经网络结构 常见的卷积神经网络结构: 服务器上:LeNet.AlexNet.VGG.InceptionV1-V4.Inception-ResNet.ResNet 手机上:SqueezNet. ...
- st-link v2怎么连接_深度学习之 YOLO v1,v2,v3详解 - 一杯清酒邀明月
(写在前面:如果你想 run 起来,立马想看看效果,那就直接跳转到最后一张,动手实践,看了结果再来往前看吧,开始吧······) 一.YOLOv1 简介 这里不再赘述,之前的我的一个 GitChat ...
- 【深度学习】ResNet的思想
转载地址:https://www.jianshu.com/p/e58437f39f65 网络的深度为什么重要? 因为CNN能够提取low/mid/high-level的特征,网络的层数越多,意味着能够 ...
最新文章
- Code Reading -chap4
- Nmap源码分析(脚本引擎)
- avrorecord.java,失败,但发生异常java.io.IOException:org.apache.avro.AvroTypeException:发现的很长,期望在配置单元中实现联合...
- c swap方法在哪个库里面_swap
- Codeforces 724 C. Ray Tracing
- iview select选中值取值_iView的Select选择框
- 2.word转换为pdf
- [修复] Eclipse – Java编译器级别与已安装的Java项目构面的版本不匹配
- [转]在C#中使用MSHTML的高级支持接口
- 高数复习:极限与连续,一元函数微分学,一元函数积分学,多元函数微分学以及微分方程(思维导图版知识点总结)
- CrystalReports2007安装包
- html5毕业设计程序,网页毕业设计制作流程
- 动手学深度学习(tensorflow)---学习笔记整理(五、过拟合和欠拟合相关问题篇)
- 用Python分析豆瓣电影Top250
- 前端(html[5],css[3])学习总结+参考手册下载
- 一起聊聊什么是P问题、NP问题、NPC问题
- jquery easyui datagrid 列自适应窗口宽度
- 人工智能 | ShowMeAI资讯日报 #2022.06.25
- 基于STM32设计智能称重系统(华为云IOT)
- 囤内存条,比买基金、白酒还赚钱