tensorflow实现深度可分离卷积

数据库：MNIST，与这里对比
tf.nn.depthwise_conv2d的理解看这里，主要是对卷积核参数的理解，即（高度，宽度，输入通道，每个通道得到的输出通道数）
训练速度慢，收敛也慢，刚开始就像没训练的样子，只将一个卷积层改成深度可分离卷积就增加了12次迭代
20211213：将第一个卷积层改成深度可分离卷积后，训练始终不收敛。所以改成下列代码，多训练了16次，模型减小了6.85%。使用sp_conv也会不收敛，可能是batch_normalization的原因
20211214：破案了，是relu的锅，注释以后，多训练了28次，看网上说好像要减小学习率

import tensorflow as tf
import numpy as np
import random
import cv2,sys,os
import MyDatadef sp_conv(name, data, kernel_size, input_num, output_num, padding, data_format='NHWC'):with tf.variable_scope(name):weight = tf.get_variable(name='weight', dtype=tf.float32, trainable=True, shape=[kernel_size,kernel_size,input_num,1], initializer=tf.random_normal_initializer(stddev=0.01))conv = tf.nn.depthwise_conv2d(data, weight, [1,1,1,1], padding, data_format=data_format)conv = tf.layers.batch_normalization(conv, momentum=0.9)#conv = tf.nn.leaky_relu(conv, alpha=0.1)point_weight = tf.get_variable(name='point_weight', dtype=tf.float32, trainable=True, shape=[1,1,input_num,output_num], initializer=tf.random_normal_initializer(stddev=0.01))conv = tf.nn.conv2d(conv, point_weight, [1,1,1,1], padding, data_format=data_format)conv = tf.layers.batch_normalization(conv, momentum=0.9)#conv = tf.nn.leaky_relu(conv, alpha=0.1)return convdata=tf.placeholder(tf.float32, [None, 28, 28, 3],name='data')
label=tf.placeholder(tf.float32, [None, 10], name='label')with tf.variable_scope('conv1'): # output is 28x28weight = tf.get_variable(name='weight', dtype=tf.float32, trainable=True, shape=[5,5,1,6], initializer=tf.random_normal_initializer(stddev=0.01))conv1 = tf.nn.conv2d(data, weight, [1,1,1,1], 'SAME')bias = tf.get_variable(name='bias', shape=6, trainable=True, dtype=tf.float32, initializer=tf.constant_initializer(0.0))conv1 = tf.nn.bias_add(conv1, bias)
with tf.variable_scope('pool1'):pool1 = tf.nn.max_pool(conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')
with tf.variable_scope('conv2'): # output is 10x10weight2 = tf.get_variable(name='weight', dtype=tf.float32, trainable=True, shape=[5,5,6,1], initializer=tf.random_normal_initializer(stddev=0.01))conv2 = tf.nn.depthwise_conv2d(pool1, weight2, [1,1,1,1], 'VALID')point_weight2 = tf.get_variable(name='point_weight', dtype=tf.float32, trainable=True, shape=[1,1,6,16], initializer=tf.random_normal_initializer(stddev=0.01))conv2 = tf.nn.conv2d(conv2, point_weight2, [1,1,1,1], 'VALID')bias2 = tf.get_variable(name='bias', shape=16, trainable=True, dtype=tf.float32, initializer=tf.constant_initializer(0.0))conv2 = tf.nn.bias_add(conv2, bias2)
with tf.variable_scope('pool2'):pool2 = tf.nn.max_pool(conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='VALID')
with tf.variable_scope('dense1'):flat = tf.reshape(pool2, [-1, 5*5*16])dense1 = tf.layers.dense(inputs=flat, units=80, activation=tf.nn.relu, use_bias=True)
with tf.variable_scope('dense2'):dense2 = tf.layers.dense(inputs=dense1, units=10, activation=None, use_bias=True)
y = tf.nn.softmax(dense2)
# loss
cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=label, logits=dense2)
loss=tf.reduce_sum(cross_entropy)
update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):train_step = tf.train.GradientDescentOptimizer(1e-4).minimize(loss)
# accuracy
correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(label, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))config = tf.ConfigProto()
config.gpu_options.allow_growth = True
config.gpu_options.per_process_gpu_memory_fraction = 0.1
sess = tf.Session(config=config)
sess.run(tf.global_variables_initializer())
input_data = MyData.Dataset('/home/lwd/data/mnist/train.txt', True, 32)
test_data = MyData.Dataset('/home/lwd/data/mnist/test.txt', False, 32)
saver = tf.train.Saver()
summary_writer = tf.summary.FileWriter('./log/', sess.graph)for i in range(100000):total = 0cnt = 0tl = 0for item in input_data:_, acc, lo = sess.run([train_step, accuracy, loss], feed_dict={data:item[0], label:item[1]})total += acccnt += 1.0tl += loprint(i, total/cnt, tl / cnt)if total/cnt > 0.88:saver.save(sess, './checkpoint/mb')xh = 0acc = 0for item in test_data:yy = sess.run(y, feed_dict={data:item[0]})for k in range(yy.shape[0]):if(np.argmax(yy[k]) == np.argmax(item[1][k])) : acc += 1xh += 1print(acc * 1.0 / xh)sys.exit(0)

tensorflow实现深度可分离卷积相关推荐

深度可分离卷积（Xception 与 MobileNet）
前言从卷积神经网络登上历史舞台开始,经过不断的改进和优化,卷积早已不是当年的卷积,诞生了分组卷积(Group convolution).空洞卷积(Dilated convolution 或 À tr ...
【Tensorflow】tf.nn.depthwise_conv2d如何实现深度卷积?+深度可分离卷积详解
目录常规卷积操作深度可分离卷积 = 逐通道卷积+逐点卷积 1.逐通道卷积 2.逐点卷积参数对比介绍实验代码清单一些轻量级的网络,如mobilenet中,会有深度可分离卷积depthwis ...
Lesson 16.1016.1116.1216.13 卷积层的参数量计算，1x1卷积核分组卷积与深度可分离卷积全连接层 nn.Sequential全局平均池化，NiN网络复现
二架构对参数量/计算量的影响在自建架构的时候,除了模型效果之外,我们还需要关注模型整体的计算效率.深度学习模型天生就需要大量数据进行训练,因此每次训练中的参数量和计算量就格外关键,因此在设计卷积网 ...
【CV】MobileNet：使用深度可分离卷积实现用于嵌入式设备的 CNN 架构
论文名称:MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications 论文下载:https:/ ...
深度学习中的depthwise convolution,pointwise convolution,SeparableConv2D深度可分离卷积
DepthwiseConv2D深度方向的空间卷积 pointwise convolution, SeparableConv2D深度可分离卷积 SeparableConv2D实现整个深度分离卷积过程,即 ...
Xception深度可分离卷积-论文笔记
Xception Xception: Deep Learning with Depthwise Separable Convolutions 角度:卷积的空间相关性和通道相关性 . 笔记还是手写好,都 ...
【深度学习】利用深度可分离卷积减小计算量及提升网络性能
[深度学习]利用深度可分离卷积减小计算量及提升网络性能文章目录 1 深度可分离卷积 2 一个深度可分离卷积层的代码示例(keras) 3 优势与创新3.1 Depthwise 过程3.2 Point ...
2d 蓝图_“蓝图”卷积--对深度可分离卷积的再思考
论文:Rethinking Depthwise Separable Convolutions: How Intra-Kernel Correlations Lead to Improved Mobil ...
深度可分离卷积Depthwise Separable Convolution
从卷积神经网络登上历史舞台开始,经过不断的改进和优化,卷积早已不是当年的卷积,诞生了分组卷积(Group convolution).空洞卷积(Dilated convolution 或 À trous ...
深度学习自学（十九）：caffe添加深度可分离卷积
下面是两种不同的深度可分离卷积的实现方式,自己在训练关键点模型,采用MobileNet 添加深度可分离卷积,发现有两种不同的可分离卷积的实现,名字不相同,但是内部都是深度可分离.DepthwiseCo ...

tensorflow实现深度可分离卷积

tensorflow实现深度可分离卷积相关推荐

最新文章

热门文章