8.1 卷积函数的使用
程序：

import tensorflow as tf# shape = [batch,in_height,in_width,in_channels]
# shape = [训练时一个批次的图片数量，图片高度，图片宽度，图像通道数]
input = tf.Variable(tf.constant(1.0,shape = [1,5,5,1]))
input2 = tf.Variable(tf.constant(1.0,shape=[1,5,5,2]))
input3 = tf.Variable(tf.constant(1.0,shape=[1,4,4,1]))# shape = [filter_height,filter_sidth,in_channels,out_channels]
# shape = [卷积核的高度，卷积核宽度，图像通道数，卷积核个数]
filter1 = tf.Variable(tf.constant([-1.0,0,0,-1],shape = [2,2,1,1]))
filter2 = tf.Variable(tf.constant([-1.0,0,0,-1,-1.0,0,0,-1],shape = [2,2,1,2]))
filter3 = tf.Variable(tf.constant([-1.0,0,0,-1,-1.0,0,0,-1,-1.0,0,0,-1],shape = [2,2,1,3]))
filter4 = tf.Variable(tf.constant([-1.0,0,0,-1,-1.0,0,0,-1,-1.0,0,0,-1,-1.0,0,0,-1],shape=[2,2,2,2]))
filter5 = tf.Variable(tf.constant([-1.0,0,0,-1,-1.0,0,0,-1],shape = [2,2,2,1]))# padding的值为VALID，表示边缘不填充；
# 当其为SAME时，表示填充到卷积核可以到达图像边缘
op1 = tf.nn.conv2d(input,filter1,strides = [1,2,2,1],padding = 'SAME')
op2 = tf.nn.conv2d(input,filter2,strides = [1,2,2,1],padding = 'SAME')
op3 = tf.nn.conv2d(input,filter3,strides = [1,2,2,1],padding = 'SAME')
op4 = tf.nn.conv2d(input2,filter4,strides = [1,2,2,1],padding = 'SAME')
op5 = tf.nn.conv2d(input2,filter5,strides = [1,2,2,1],padding = 'SAME')
vop1 = tf.nn.conv2d(input,filter1,strides = [1,2,2,1],padding = 'VALID')
op6 = tf.nn.conv2d(input3,filter1,strides = [1,2 ,2,1],padding = 'SAME')
vop6 = tf.nn.conv2d(input3,filter1,strides = [1,2,2,1],padding = 'VALID')init = tf.global_variables_initializer()
with tf.Session() as sess:sess.run(init)print("op1:\n",sess.run([op1,filter1]))print("----------------------")print("op2:\n", sess.run([op2, filter2]))print("op3:\n", sess.run([op3, filter3]))print("----------------------")print("op4:\n", sess.run([op4, filter4]))print("op5:\n", sess.run([op5, filter5]))print("----------------------")print("op1:\n", sess.run([op1, filter1]))print("vop1:\n", sess.run([vop1, filter1]))print("op6:\n", sess.run([op6, filter1]))print("vop6:\n", sess.run([vop6, filter1]))

结果：

op1:[array([[[[-2.],[-2.],[-1.]],[[-2.],[-2.],[-1.]],[[-1.],[-1.],[-1.]]]], dtype=float32), array([[[[-1.]],[[ 0.]]],[[[ 0.]],[[-1.]]]], dtype=float32)]
----------------------
op2:[array([[[[-2., -2.],[-2., -2.],[-2.,  0.]],[[-2., -2.],[-2., -2.],[-2.,  0.]],[[-1., -1.],[-1., -1.],[-1.,  0.]]]], dtype=float32), array([[[[-1.,  0.]],[[ 0., -1.]]],[[[-1.,  0.]],[[ 0., -1.]]]], dtype=float32)]
op3:[array([[[[-2., -2., -2.],[-2., -2., -2.],[-1., -1., -1.]],[[-2., -2., -2.],[-2., -2., -2.],[-1., -1., -1.]],[[-2., -1.,  0.],[-2., -1.,  0.],[-1.,  0.,  0.]]]], dtype=float32), array([[[[-1.,  0.,  0.]],[[-1., -1.,  0.]]],[[[ 0., -1., -1.]],[[ 0.,  0., -1.]]]], dtype=float32)]
----------------------
op4:[array([[[[-4., -4.],[-4., -4.],[-2., -2.]],[[-4., -4.],[-4., -4.],[-2., -2.]],[[-2., -2.],[-2., -2.],[-1., -1.]]]], dtype=float32), array([[[[-1.,  0.],[ 0., -1.]],[[-1.,  0.],[ 0., -1.]]],[[[-1.,  0.],[ 0., -1.]],[[-1.,  0.],[ 0., -1.]]]], dtype=float32)]
op5:[array([[[[-4.],[-4.],[-2.]],[[-4.],[-4.],[-2.]],[[-2.],[-2.],[-1.]]]], dtype=float32), array([[[[-1.],[ 0.]],[[ 0.],[-1.]]],[[[-1.],[ 0.]],[[ 0.],[-1.]]]], dtype=float32)]
----------------------
op1:[array([[[[-2.],[-2.],[-1.]],[[-2.],[-2.],[-1.]],[[-1.],[-1.],[-1.]]]], dtype=float32), array([[[[-1.]],[[ 0.]]],[[[ 0.]],[[-1.]]]], dtype=float32)]
vop1:[array([[[[-2.],[-2.]],[[-2.],[-2.]]]], dtype=float32), array([[[[-1.]],[[ 0.]]],[[[ 0.]],[[-1.]]]], dtype=float32)]
op6:[array([[[[-2.],[-2.]],[[-2.],[-2.]]]], dtype=float32), array([[[[-1.]],[[ 0.]]],[[[ 0.]],[[-1.]]]], dtype=float32)]
vop6:[array([[[[-2.],[-2.]],[[-2.],[-2.]]]], dtype=float32), array([[[[-1.]],[[ 0.]]],[[[ 0.]],[[-1.]]]], dtype=float32)]

8.2 sobel算子
通过卷积操作实现sobel算子，讲彩色图片生成带有边缘化信息的图片
程序：

import matplotlib.pyplot as plt
import matplotlibimport numpy as np
import tensorflow as tfmying = matplotlib.image.imread("0.png")
plt.imshow(mying)
plt.axis('off')
plt.show()
print(mying.shape)full = np.reshape(mying,[1,264,256,3])
inputfull = tf.Variable(tf.constant(1.0,shape = [1,264,256,3]))
filter = tf.Variable(tf.constant([[-1.0,-1.0,-1.0],[0,0,0],[1.0,1.0,1.0],[-2.0,-2.0,-2.0],[0,0,0],[2.0,2.0,2.0],[-1.0,-1.0,-1.0],[0,0,0],[1.0,1.0,1.0]],shape = [3,3,3,1]))
op = tf.nn.conv2d(inputfull,filter,strides=[1,1,1,1],padding='SAME')
# 3个通道输入，生成1个feature ma
o =tf.cast(((op -tf.reduce_min(op))/(tf.reduce_max(op)-tf.reduce_min(op)))*255,tf.uint8)with tf.Session() as sess:sess.run(tf.global_variables_initializer())t,f = sess.run([o,filter],feed_dict={inputfull:full})t = np.reshape(t,[264,256])plt.imshow(t,cmap='Greys_r')plt.axis('off')plt.show()

原图：

结果图：

8.3 池化函数
池化（为了降维）。均值池化：对里面所有不为0的像素点取均值，这种方法得到的特征数据会对背景信息更敏感一些；最大池化：所有像素点取最大值。
定义1个输入变量来模拟输入图片，4*4大小的2通道矩阵，并将其赋予指定的值。2个通道分别为4个0到4个3组成的矩阵，4个4到4个7组成的矩阵。

程序：

import tensorflow as tf#1 定义输入变量
img = tf.constant([[[0.0, 4.0], [0.0, 4.0], [0.0, 4.0], [0.0, 4.0]],[[1.0, 5.0], [1.0, 5.0], [1.0, 5.0], [1.0, 5.0]],[[2.0, 6.0], [2.0, 6.0], [2.0, 6.0], [2.0, 6.0]],[[3.0, 7.0], [3.0, 7.0], [3.0, 7.0], [3.0, 7.0]]
])img = tf.reshape(img, [1, 4, 4, 2])#2 定义池化操作
pooling = tf.nn.max_pool(img, [1, 2, 2, 1], [1, 2, 2, 1], padding='VALID')
pooling1 = tf.nn.max_pool(img, [1, 2, 2, 1], [1, 1, 1, 1], padding='VALID')
pooling2 = tf.nn.avg_pool(img, [1, 4, 4, 1], [1, 1, 1, 1], padding='SAME')
pooling3 = tf.nn.avg_pool(img, [1, 4, 4, 1], [1, 4, 4, 1], padding='SAME')
nt_hpool2_flat = tf.reshape(tf.transpose(img), [-1, 16])
pooling4 = tf.reduce_mean(nt_hpool2_flat, 1)  # 1对行求均值（1表示轴是列）   0 对列求均值#3 运行池化操作
with tf.Session() as sess:print("image:")image = sess.run(img)print(image)result = sess.run(pooling)print("reslut:\n", result)result = sess.run(pooling1)print("reslut1:\n", result)result = sess.run(pooling2)print("reslut2:\n", result)result = sess.run(pooling3)print("reslut3:\n", result)flat, result = sess.run([nt_hpool2_flat, pooling4])print("reslut4:\n", result)print("flat:\n", flat)

结果：

image:
[[[[0. 4.][0. 4.][0. 4.][0. 4.]][[1. 5.][1. 5.][1. 5.][1. 5.]][[2. 6.][2. 6.][2. 6.][2. 6.]][[3. 7.][3. 7.][3. 7.][3. 7.]]]]
reslut:[[[[1. 5.][1. 5.]][[3. 7.][3. 7.]]]]
reslut1:[[[[1. 5.][1. 5.][1. 5.]][[2. 6.][2. 6.][2. 6.]][[3. 7.][3. 7.][3. 7.]]]]
reslut2:[[[[1.  5. ][1.  5. ][1.  5. ][1.  5. ]][[1.5 5.5][1.5 5.5][1.5 5.5][1.5 5.5]][[2.  6. ][2.  6. ][2.  6. ][2.  6. ]][[2.5 6.5][2.5 6.5][2.5 6.5][2.5 6.5]]]]
reslut3:[[[[1.5 5.5]]]]
reslut4:[1.5 5.5]
flat:[[0. 1. 2. 3. 0. 1. 2. 3. 0. 1. 2. 3. 0. 1. 2. 3.][4. 5. 6. 7. 4. 5. 6. 7. 4. 5. 6. 7. 4. 5. 6. 7.]]

8.6 cifar手动读取

程序：

import numpy as np
# 将指定的文件放到指定的目录下
filename = 'D:/tmp/cifar10_data/cifar-10-batches-bin/test_batch.bin'bytestream = open(filename, "rb")
buf = bytestream.read(10000 * (1 + 32 * 32 * 3))
bytestream.close()data = np.frombuffer(buf, dtype=np.uint8)
data = data.reshape(10000, 1 + 32 * 32 * 3)
labels_images = np.hsplit(data, [1])
labels = labels_images[0].reshape(10000)
images = labels_images[1].reshape(10000, 32, 32, 3)img = np.reshape(images[0], (3, 32, 32))  # 导出第一幅图
img = img.transpose(1, 2, 0)import pylabprint(labels[0])
pylab.imshow(img)
pylab.show()

结果：

8.7 使用队列机制读取数据queue
提供一个队列机制，tf.train.start_queue_runners()是启动线程，向队列里面读取数据。同时需要改变程序，并且增加一个协调器，以信号量的方式来协调线程间的关系，完成线程间的同步。
由于使用了Coordinator，当session要关闭之前会进行coord.request_stop函数将所有线程关闭，之后才会关闭session。

程序：

import tensorflow as tf# 创建长度为100的队列
queue = tf.FIFOQueue(100, "float")c = tf.Variable(0.0)  # 计数器
# 加1操作
op = tf.assign_add(c, tf.constant(1.0))
# 操作:将计数器的结果加入队列
enqueue_op = queue.enqueue(c)# 创建一个队列管理器QueueRunner，用这两个操作向q中添加元素。
# 目前我们只使用一个线程:
qr = tf.train.QueueRunner(queue, enqueue_ops=[op, enqueue_op])with tf.Session() as sess:sess.run(tf.global_variables_initializer())coord = tf.train.Coordinator()## 启动入队线程, Coordinator是线程的参数enqueue_threads = qr.create_threads(sess, coord=coord, start=True)  # 启动入队线程# 主线程for i in range(0, 10):print("-------------------------")print(sess.run(queue.dequeue()))coord.request_stop()  # 通知其他线程关闭 其他所有线程关闭之后，这一函数才能返回# join操作经常用在线程当中,其作用是等待某线程结束# coord.join(enqueue_threads)

结果：

820.0
-------------------------
828.0
-------------------------
840.0
-------------------------
850.0
-------------------------
857.0
-------------------------
866.0
-------------------------
878.0
-------------------------
890.0
-------------------------
900.0
-------------------------
909.0

8.9 cifar卷积
程序：

# 将cifar10.py和cifar10_input.py函数加入你引用的库中。
# 将需要引用的文件放入需要引用的文件夹（此文件夹放Project中即可）中，
# 选择file-setting-Project Structure
# 右击你需要引用的文件夹，选择Sources选项，此时文件将被引用import cifar10_input
import tensorflow as tf
import numpy as npbatch_size = 128
# 将data_dir的目标改成解压后的素材
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.1)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')def avg_pool_6x6(x):return tf.nn.avg_pool(x, ksize=[1, 6, 6, 1],strides=[1, 6, 6, 1], padding='SAME')# tf Graph Input
x = tf.placeholder(tf.float32, [None, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classesW_conv1 = weight_variable([5, 5, 3, 64])
b_conv1 = bias_variable([64])x_image = tf.reshape(x, [-1, 24, 24, 3])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)W_conv2 = weight_variable([5, 5, 64, 64])
b_conv2 = bias_variable([64])h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)nt_hpool3 = avg_pool_6x6(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.train.start_queue_runners(sess=sess)
for i in range(15000):  # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b}, session=sess)if i % 200 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))

结果：

step 0, training accuracy 0.078125
step 200, training accuracy 0.359375
step 400, training accuracy 0.367188
step 600, training accuracy 0.382812
step 800, training accuracy 0.445312
step 1000, training accuracy 0.460938
step 1200, training accuracy 0.453125
step 1400, training accuracy 0.5
step 1600, training accuracy 0.421875
step 1800, training accuracy 0.429688
step 2000, training accuracy 0.59375
step 2200, training accuracy 0.554688
step 2400, training accuracy 0.601562
step 2600, training accuracy 0.515625
step 2800, training accuracy 0.546875
step 3000, training accuracy 0.59375
step 3200, training accuracy 0.507812
step 3400, training accuracy 0.570312
step 3600, training accuracy 0.5625
step 3800, training accuracy 0.625
step 4000, training accuracy 0.554688
step 4200, training accuracy 0.507812
step 4400, training accuracy 0.59375
step 4600, training accuracy 0.5625
step 4800, training accuracy 0.609375
step 5000, training accuracy 0.578125
step 5200, training accuracy 0.601562
step 5400, training accuracy 0.664062
step 5600, training accuracy 0.59375
step 5800, training accuracy 0.65625
step 6000, training accuracy 0.585938
step 6200, training accuracy 0.664062
step 6400, training accuracy 0.609375
step 6600, training accuracy 0.640625
step 6800, training accuracy 0.59375
step 7000, training accuracy 0.65625
step 7200, training accuracy 0.625
step 7400, training accuracy 0.648438
step 7600, training accuracy 0.671875
step 7800, training accuracy 0.59375
step 8000, training accuracy 0.59375
step 8200, training accuracy 0.585938
step 8400, training accuracy 0.617188
step 8600, training accuracy 0.59375
step 8800, training accuracy 0.625
step 9000, training accuracy 0.671875
step 9200, training accuracy 0.617188
step 9400, training accuracy 0.632812
step 9600, training accuracy 0.648438
step 9800, training accuracy 0.710938
step 10000, training accuracy 0.664062
step 10200, training accuracy 0.664062
step 10400, training accuracy 0.71875
step 10600, training accuracy 0.625
step 10800, training accuracy 0.640625
step 11000, training accuracy 0.679688
step 11200, training accuracy 0.75
step 11400, training accuracy 0.648438
step 11600, training accuracy 0.648438
step 11800, training accuracy 0.609375
step 12000, training accuracy 0.71875
step 12200, training accuracy 0.695312
step 12400, training accuracy 0.609375
step 12600, training accuracy 0.742188
step 12800, training accuracy 0.65625
step 13000, training accuracy 0.664062
step 13200, training accuracy 0.640625
step 13400, training accuracy 0.640625
step 13600, training accuracy 0.570312
step 13800, training accuracy 0.703125
step 14000, training accuracy 0.695312
step 14200, training accuracy 0.65625
step 14400, training accuracy 0.679688
step 14600, training accuracy 0.71875
step 14800, training accuracy 0.640625
finished！ test accuracy 0.59375

8.12 反卷积操作
程序：

import tensorflow as tfimg = tf.Variable(tf.constant(1.0, shape=[1, 4, 4, 1]))filter = tf.Variable(tf.constant([1.0, 0, -1, -2], shape=[2, 2, 1, 1]))conv = tf.nn.conv2d(img, filter, strides=[1, 2, 2, 1], padding='VALID')
cons = tf.nn.conv2d(img, filter, strides=[1, 2, 2, 1], padding='SAME')
print(conv.shape)
print(cons.shape)contv = tf.nn.conv2d_transpose(conv, filter, [1, 4, 4, 1], strides=[1, 2, 2, 1], padding='VALID')
conts = tf.nn.conv2d_transpose(cons, filter, [1, 4, 4, 1], strides=[1, 2, 2, 1], padding='SAME')with tf.Session() as sess:sess.run(tf.global_variables_initializer())print("conv:\n", sess.run([conv, filter]))print("cons:\n", sess.run([cons]))print("contv:\n", sess.run([contv]))print("conts:\n", sess.run([conts]))

结果：

(1, 2, 2, 1)
(1, 2, 2, 1)conv:[array([[[[-2.],[-2.]],[[-2.],[-2.]]]], dtype=float32), array([[[[ 1.]],[[ 0.]]],[[[-1.]],[[-2.]]]], dtype=float32)]
cons:[array([[[[-2.],[-2.]],[[-2.],[-2.]]]], dtype=float32)]
contv:[array([[[[-2.],[ 0.],[-2.],[ 0.]],[[ 2.],[ 4.],[ 2.],[ 4.]],[[-2.],[ 0.],[-2.],[ 0.]],[[ 2.],[ 4.],[ 2.],[ 4.]]]], dtype=float32)]
conts:[array([[[[-2.],[ 0.],[-2.],[ 0.]],[[ 2.],[ 4.],[ 2.],[ 4.]],[[-2.],[ 0.],[-2.],[ 0.]],[[ 2.],[ 4.],[ 2.],[ 4.]]]], dtype=float32)]

8.13 反池化操作

程序：

import tensorflow as tfdef max_pool_with_argmax(net, stride):_, mask = tf.nn.max_pool_with_argmax(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1],padding='SAME')mask = tf.stop_gradient(mask)net = tf.nn.max_pool(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1], padding='SAME')return net, maskdef unpool(net, mask, stride):ksize = [1, stride, stride, 1]input_shape = net.get_shape().as_list()#  calculation new shapeoutput_shape = (input_shape[0], input_shape[1] * ksize[1], input_shape[2] * ksize[2], input_shape[3])# calculation indices for batch, height, width and feature mapsone_like_mask = tf.ones_like(mask)batch_range = tf.reshape(tf.range(output_shape[0], dtype=tf.int64), shape=[input_shape[0], 1, 1, 1])b = one_like_mask * batch_rangey = mask // (output_shape[2] * output_shape[3])x = mask % (output_shape[2] * output_shape[3]) // output_shape[3]feature_range = tf.range(output_shape[3], dtype=tf.int64)f = one_like_mask * feature_range# transpose indices & reshape update values to one dimensionupdates_size = tf.size(net)indices = tf.transpose(tf.reshape(tf.stack([b, y, x, f]), [4, updates_size]))values = tf.reshape(net, [updates_size])ret = tf.scatter_nd(indices, values, output_shape)return retimg = tf.constant([[[0.0, 4.0], [0.0, 4.0], [0.0, 4.0], [0.0, 4.0]],[[1.0, 5.0], [1.0, 5.0], [1.0, 5.0], [1.0, 5.0]],[[2.0, 6.0], [2.0, 6.0], [2.0, 6.0], [2.0, 6.0]],[[3.0, 7.0], [3.0, 7.0], [3.0, 7.0], [3.0, 7.0]]
])img = tf.reshape(img, [1, 4, 4, 2])
pooling2 = tf.nn.max_pool(img, [1, 2, 2, 1], [1, 2, 2, 1], padding='SAME')
encode, mask = max_pool_with_argmax(img, 2)
img2 = unpool(encode, mask, 2)
print(img.shape)
print(encode.shape)
print(mask.shape)
print(img2.shape)
with tf.Session() as sess:print("image:")print(sess.run(img))result = sess.run(pooling2)print("pooling2:\n", result)result, mask2 = sess.run([encode, mask])print("encode:\n", result, mask2)result = sess.run(img2)print("reslut:\n", result)

结果：

(1, 4, 4, 2)
(1, 2, 2, 2)
(1, 2, 2, 2)
(1, 4, 4, 2)
[[[[0. 4.][0. 4.][0. 4.][0. 4.]][[1. 5.][1. 5.][1. 5.][1. 5.]][[2. 6.][2. 6.][2. 6.][2. 6.]][[3. 7.][3. 7.][3. 7.][3. 7.]]]]
pooling2:[[[[1. 5.][1. 5.]][[3. 7.][3. 7.]]]]
encode:[[[[1. 5.][1. 5.]][[3. 7.][3. 7.]]]] [[[[ 8  9][12 13]][[24 25][28 29]]]]
reslut:[[[[0. 0.][0. 0.][0. 0.][0. 0.]][[1. 5.][0. 0.][1. 5.][0. 0.]][[0. 0.][0. 0.][0. 0.][0. 0.]][[3. 7.][0. 0.][3. 7.][0. 0.]]]]

8.15 gradients梯度求解
程序：

import tensorflow as tftf.reset_default_graph()
w1 = tf.get_variable('w1', shape=[2])
w2 = tf.get_variable('w2', shape=[2])w3 = tf.get_variable('w3', shape=[2])
w4 = tf.get_variable('w4', shape=[2])y1 = w1 + w2 + w3
y2 = w3 + w4a = w1 + w2
a_stoped = tf.stop_gradient(a)
y3 = a_stoped + w3gradients = tf.gradients([y1, y2], [w1, w2, w3, w4], grad_ys=[tf.convert_to_tensor([1., 2.]),tf.convert_to_tensor([3., 4.])])
with tf.Session() as sess:sess.run(tf.global_variables_initializer())print(sess.run(gradients))

结果：

[array([1., 2.], dtype=float32), array([1., 2.], dtype=float32), array([4., 6.], dtype=float32), array([3., 4.], dtype=float32)]

8.16 gradients设置梯度停止

程序：

import tensorflow as tftf.reset_default_graph()
w1 = tf.get_variable('w1', shape=[2])
w2 = tf.get_variable('w2', shape=[2])w3 = tf.get_variable('w3', shape=[2])
w4 = tf.get_variable('w4', shape=[2])y1 = w1 + w2 + w3
y2 = w3 + w4a = w1 + w2
# 对于反向传播过程中某种特殊情况需要停止梯度的运算时使用此函数，
# 被他定义过的节点将没有梯度运算功能
a_stoped = tf.stop_gradient(a)
y3 = a_stoped + w3gradients = tf.gradients([y1, y2], [w1, w2, w3, w4], grad_ys=[tf.convert_to_tensor([1., 2.]),tf.convert_to_tensor([3., 4.])])gradients2 = tf.gradients(y3, [w1, w2, w3], grad_ys=tf.convert_to_tensor([1., 2.]))
print(gradients2)gradients3 = tf.gradients(y3, [w3], grad_ys=tf.convert_to_tensor([1., 2.]))with tf.Session() as sess:sess.run(tf.global_variables_initializer())print(sess.run(gradients))# print(sess.run(gradients2))#报错print(sess.run(gradients3))

结果：

[None, None, <tf.Tensor 'gradients_1/grad_ys_0:0' shape=(2,) dtype=float32>]
[array([1., 2.], dtype=float32), array([1., 2.], dtype=float32), array([4., 6.], dtype=float32), array([3., 4.], dtype=float32)]
[array([1., 2.], dtype=float32)]

8.17 cifar反卷积

程序：

import cifar10_input
import tensorflow as tf
import numpy as np# 最大池化
def max_pool_with_argmax(net, stride):_, mask = tf.nn.max_pool_with_argmax(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1],padding='SAME')mask = tf.stop_gradient(mask)net = tf.nn.max_pool(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1], padding='SAME')return net, mask# 4*4----2*2--=2*2 【6，8，12，16】
# 反池化
def unpool(net, mask, stride):ksize = [1, stride, stride, 1]input_shape = net.get_shape().as_list()output_shape = (input_shape[0], input_shape[1] * ksize[1], input_shape[2] * ksize[2], input_shape[3])one_like_mask = tf.ones_like(mask)batch_range = tf.reshape(tf.range(output_shape[0], dtype=tf.int64), shape=[input_shape[0], 1, 1, 1])b = one_like_mask * batch_rangey = mask // (output_shape[2] * output_shape[3])x = mask % (output_shape[2] * output_shape[3]) // output_shape[3]feature_range = tf.range(output_shape[3], dtype=tf.int64)f = one_like_mask * feature_rangeupdates_size = tf.size(net)indices = tf.transpose(tf.reshape(tf.stack([b, y, x, f]), [4, updates_size]))values = tf.reshape(net, [updates_size])ret = tf.scatter_nd(indices, values, output_shape)return retbatch_size = 128
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.01)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.01, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')def avg_pool_6x6(x):return tf.nn.avg_pool(x, ksize=[1, 6, 6, 1],strides=[1, 6, 6, 1], padding='SAME')# tf Graph Input
x = tf.placeholder(tf.float32, [batch_size, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [batch_size, 10])  # 0-9 数字=> 10 classesW_conv1 = weight_variable([5, 5, 3, 64])
b_conv1 = bias_variable([64])x_image = tf.reshape(x, [-1, 24, 24, 3])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
# h_pool1 = max_pool_2x2(h_conv1)
h_pool1, mask1 = max_pool_with_argmax(h_conv1, 2)W_conv2 = weight_variable([5, 5, 64, 64])
b_conv2 = bias_variable([64])h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
# h_pool2 = max_pool_2x2(h_conv2)#############################################################
h_pool2, mask = max_pool_with_argmax(h_conv2, 2)  # (128, 6, 6, 64)
print(h_pool2.shape)
t_conv2 = unpool(h_pool2, mask, 2)  # (128, 12, 12, 64)
t_pool1 = tf.nn.conv2d_transpose(t_conv2 - b_conv2, W_conv2, h_pool1.shape, [1, 1, 1, 1])  # (128, 24, 24, 64)
print(t_conv2.shape, h_pool1.shape, t_pool1.shape)
t_conv1 = unpool(t_pool1, mask1, 2)
t_x_image = tf.nn.conv2d_transpose(t_conv1 - b_conv1, W_conv1, x_image.shape, [1, 1, 1, 1])# 第一层卷积还原
t1_conv1 = unpool(h_pool1, mask1, 2)
t1_x_image = tf.nn.conv2d_transpose(t1_conv1 - b_conv1, W_conv1, x_image.shape, [1, 1, 1, 1])# 生成最终图像
stitched_decodings = tf.concat((x_image, t1_x_image, t_x_image), axis=2)
decoding_summary_op = tf.summary.image('source/cifar', stitched_decodings)#############################################################W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)nt_hpool3 = avg_pool_6x6(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)cross_entropy = -tf.reduce_sum(y * tf.log(y_conv)) + (tf.nn.l2_loss(W_conv1) + tf.nn.l2_loss(W_conv2) + tf.nn.l2_loss(W_conv3))train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
summary_writer = tf.summary.FileWriter('./log/', sess.graph)tf.train.start_queue_runners(sess=sess)for i in range(15000):  # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b}, session=sess)# _, decoding_summary = sess.run([train_step, decoding_summary_op],feed_dict={x:image_batch, y: label_b})if i % 200 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))print("cross_entropy", cross_entropy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))# summary_writer.add_summary(decoding_summary)image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))
decoding_summary = sess.run(decoding_summary_op, feed_dict={x: image_batch, y: label_b})
summary_writer.add_summary(decoding_summary)

结果：

step 0, training accuracy 0.171875
cross_entropy 299.3061
step 200, training accuracy 0.265625
cross_entropy 242.04913
step 400, training accuracy 0.320312
cross_entropy 235.10938
step 600, training accuracy 0.328125
cross_entropy 236.76186
step 800, training accuracy 0.40625
cross_entropy 218.89662
step 1000, training accuracy 0.382812
cross_entropy 219.23944
step 1200, training accuracy 0.40625
cross_entropy 207.38719
step 1400, training accuracy 0.390625
cross_entropy 213.67719
step 1600, training accuracy 0.304688
cross_entropy 230.81429
step 1800, training accuracy 0.359375
cross_entropy 227.84703
step 2000, training accuracy 0.46875
cross_entropy 215.76927
step 2200, training accuracy 0.484375
cross_entropy 185.36913
step 2400, training accuracy 0.421875
cross_entropy 209.09827
step 2600, training accuracy 0.554688
cross_entropy 180.51645
step 2800, training accuracy 0.476562
cross_entropy 192.39822
step 3000, training accuracy 0.4375
cross_entropy 197.83093
step 3200, training accuracy 0.445312
cross_entropy 210.26898
step 3400, training accuracy 0.523438
cross_entropy 188.47134
step 3600, training accuracy 0.492188
cross_entropy 192.4504
step 3800, training accuracy 0.5
cross_entropy 179.52985
step 4000, training accuracy 0.5
cross_entropy 187.8034
step 4200, training accuracy 0.515625
cross_entropy 178.45355
step 4400, training accuracy 0.53125
cross_entropy 183.52817
step 4600, training accuracy 0.578125
cross_entropy 173.862
step 4800, training accuracy 0.515625
cross_entropy 184.0677
step 5000, training accuracy 0.546875
cross_entropy 178.71603
step 5200, training accuracy 0.507812
cross_entropy 185.81891
step 5400, training accuracy 0.578125
cross_entropy 169.00998
step 5600, training accuracy 0.445312
cross_entropy 202.5278
step 5800, training accuracy 0.523438
cross_entropy 184.44104
step 6000, training accuracy 0.53125
cross_entropy 188.50015
step 6200, training accuracy 0.632812
cross_entropy 154.6417
step 6400, training accuracy 0.5625
cross_entropy 164.94344
step 6600, training accuracy 0.476562
cross_entropy 191.15407
step 6800, training accuracy 0.523438
cross_entropy 183.11644
step 7000, training accuracy 0.601562
cross_entropy 156.16698
step 7200, training accuracy 0.554688
cross_entropy 179.59787
step 7400, training accuracy 0.585938
cross_entropy 161.66383
step 7600, training accuracy 0.507812
cross_entropy 194.35725
step 7800, training accuracy 0.539062
cross_entropy 175.424
step 8000, training accuracy 0.578125
cross_entropy 174.82635
step 8200, training accuracy 0.53125
cross_entropy 170.75404
step 8400, training accuracy 0.554688
cross_entropy 180.11963
step 8600, training accuracy 0.585938
cross_entropy 183.36034
step 8800, training accuracy 0.578125
cross_entropy 172.72717
step 9000, training accuracy 0.609375
cross_entropy 165.07404
step 9200, training accuracy 0.601562
cross_entropy 166.86453
step 9400, training accuracy 0.617188
cross_entropy 173.84659
step 9600, training accuracy 0.585938
cross_entropy 161.06015
step 9800, training accuracy 0.601562
cross_entropy 169.96394
step 10000, training accuracy 0.59375
cross_entropy 171.04904
step 10200, training accuracy 0.5625
cross_entropy 181.28479
step 10400, training accuracy 0.609375
cross_entropy 161.9608
step 10600, training accuracy 0.578125
cross_entropy 183.9913
step 10800, training accuracy 0.664062
cross_entropy 167.06818
step 11000, training accuracy 0.53125
cross_entropy 180.94598
step 11200, training accuracy 0.515625
cross_entropy 181.30446
step 11400, training accuracy 0.554688
cross_entropy 172.07529
step 11600, training accuracy 0.539062
cross_entropy 173.89505
step 11800, training accuracy 0.570312
cross_entropy 171.61345
step 12000, training accuracy 0.632812
cross_entropy 163.43626
step 12200, training accuracy 0.6875
cross_entropy 142.34613
step 12400, training accuracy 0.640625
cross_entropy 151.77791
step 12600, training accuracy 0.640625
cross_entropy 152.30363
step 12800, training accuracy 0.59375
cross_entropy 167.594
step 13000, training accuracy 0.632812
cross_entropy 168.27171
step 13200, training accuracy 0.570312
cross_entropy 167.29607
step 13400, training accuracy 0.609375
cross_entropy 165.10593
step 13600, training accuracy 0.554688
cross_entropy 171.7725
step 13800, training accuracy 0.640625
cross_entropy 151.95393
step 14000, training accuracy 0.65625
cross_entropy 157.50107
step 14200, training accuracy 0.59375
cross_entropy 161.49493
step 14400, training accuracy 0.609375
cross_entropy 161.87918
step 14600, training accuracy 0.617188
cross_entropy 150.59903
step 14800, training accuracy 0.617188
cross_entropy 165.2901
finished！ test accuracy 0.59375

8.18 cifar简介代码

程序：

import cifar10_input
import tensorflow as tf
import numpy as npbatch_size = 128
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")# tf Graph Input
x = tf.placeholder(tf.float32, [None, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classesx_image = tf.reshape(x, [-1, 24, 24, 3])h_conv1 = tf.contrib.layers.conv2d(x_image, 64, [5, 5], 1, 'SAME', activation_fn=tf.nn.relu)
h_pool1 = tf.contrib.layers.max_pool2d(h_conv1, [2, 2], stride=2, padding='SAME')h_conv2 = tf.contrib.layers.conv2d(h_pool1, 64, [5, 5], 1, 'SAME', activation_fn=tf.nn.relu)
h_pool2 = tf.contrib.layers.max_pool2d(h_conv2, [2, 2], stride=2, padding='SAME')nt_hpool2 = tf.contrib.layers.avg_pool2d(h_pool2, [6, 6], stride=6, padding='SAME')nt_hpool2_flat = tf.reshape(nt_hpool2, [-1, 64])y_conv = tf.contrib.layers.fully_connected(nt_hpool2_flat, 10, activation_fn=tf.nn.softmax)cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.train.start_queue_runners(sess=sess)
for i in range(1000):  # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b}, session=sess)if i % 200 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))

结果：

step 0, training accuracy 0.0859375
step 200, training accuracy 0.273438
step 400, training accuracy 0.375
step 600, training accuracy 0.289062
step 800, training accuracy 0.445312
finished！ test accuracy 0.421875

8.19 cifar卷积核优化

程序：

import cifar10_input
import tensorflow as tf
import numpy as npbatch_size = 128
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.1)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')def avg_pool_6x6(x):return tf.nn.avg_pool(x, ksize=[1, 6, 6, 1],strides=[1, 6, 6, 1], padding='SAME')# tf Graph Input
x = tf.placeholder(tf.float32, [None, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classesW_conv1 = weight_variable([5, 5, 3, 64])
b_conv1 = bias_variable([64])x_image = tf.reshape(x, [-1, 24, 24, 3])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
#########################################################new########
W_conv21 = weight_variable([5, 1, 64, 64])
b_conv21 = bias_variable([64])
h_conv21 = tf.nn.relu(conv2d(h_pool1, W_conv21) + b_conv21)W_conv2 = weight_variable([1, 5, 64, 64])
b_conv2 = bias_variable([64])
h_conv2 = tf.nn.relu(conv2d(h_conv21, W_conv2) + b_conv2)
###########################################################old#########
# W_conv2 = weight_variable([5, 5, 64, 64])
# b_conv2 = bias_variable([64])
# h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
###################################################################h_pool2 = max_pool_2x2(h_conv2)
W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)nt_hpool3 = avg_pool_6x6(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.train.start_queue_runners(sess=sess)
for i in range(1000):  # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b}, session=sess)if i % 200 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))

结果：

step 0, training accuracy 0.15625
step 200, training accuracy 0.3125
step 400, training accuracy 0.375
step 600, training accuracy 0.40625
step 800, training accuracy 0.453125
finished！ test accuracy 0.414062

8.20 cifar多通道卷积

程序：

import cifar10_input
import tensorflow as tf
import numpy as npbatch_size = 128
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.1)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')def avg_pool_6x6(x):return tf.nn.avg_pool(x, ksize=[1, 6, 6, 1],strides=[1, 6, 6, 1], padding='SAME')# tf Graph Input
x = tf.placeholder(tf.float32, [None, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classesW_conv1 = weight_variable([5, 5, 3, 64])
b_conv1 = bias_variable([64])x_image = tf.reshape(x, [-1, 24, 24, 3])h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
#######################################################多卷积核
W_conv2_5x5 = weight_variable([5, 5, 64, 64])
b_conv2_5x5 = bias_variable([64])
W_conv2_7x7 = weight_variable([7, 7, 64, 64])
b_conv2_7x7 = bias_variable([64])W_conv2_3x3 = weight_variable([3, 3, 64, 64])
b_conv2_3x3 = bias_variable([64])W_conv2_1x1 = weight_variable([1, 1, 64, 64])
b_conv2_1x1 = bias_variable([64])h_conv2_1x1 = tf.nn.relu(conv2d(h_pool1, W_conv2_1x1) + b_conv2_1x1)
h_conv2_3x3 = tf.nn.relu(conv2d(h_pool1, W_conv2_3x3) + b_conv2_3x3)
h_conv2_5x5 = tf.nn.relu(conv2d(h_pool1, W_conv2_5x5) + b_conv2_5x5)
h_conv2_7x7 = tf.nn.relu(conv2d(h_pool1, W_conv2_7x7) + b_conv2_7x7)
h_conv2 = tf.concat([h_conv2_5x5, h_conv2_7x7, h_conv2_3x3, h_conv2_1x1], 3)#######################################################
# W_conv2 = weight_variable([5, 5, 64, 64])
# b_conv2 = bias_variable([64])
#
# h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
#######################################################W_conv3 = weight_variable([5, 5, 256, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)nt_hpool3 = avg_pool_6x6(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))# 不同的优化方法测测效果
# train_step = tf.train.GradientDescentOptimizer(1e-3).minimize(cross_entropy)
# train_step = tf.train.AdagradOptimizer(1e-5).minimize(cross_entropy)
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.train.start_queue_runners(sess=sess)
for i in range(1000):  # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b}, session=sess)if i % 200 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))

结果：

step 0, training accuracy 0.0703125
step 200, training accuracy 0.328125
step 400, training accuracy 0.40625
step 600, training accuracy 0.359375
step 800, training accuracy 0.460938
finished！ test accuracy 0.476562

8.21 cifarBN

程序：

import cifar10_input
import tensorflow as tf
import numpy as np
from tensorflow.contrib.layers.python.layers import batch_normbatch_size = 128
data_dir = 'D:/tmp/cifar10_data/cifar-10-batches-bin'
print("begin")
images_train, labels_train = cifar10_input.inputs(eval_data=False, data_dir=data_dir, batch_size=batch_size)
images_test, labels_test = cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)
print("begin data")def weight_variable(shape):initial = tf.truncated_normal(shape, stddev=0.1)return tf.Variable(initial)def bias_variable(shape):initial = tf.constant(0.1, shape=shape)return tf.Variable(initial)def conv2d(x, W):return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')def max_pool_2x2(x):return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],strides=[1, 2, 2, 1], padding='SAME')def avg_pool_6x6(x):return tf.nn.avg_pool(x, ksize=[1, 6, 6, 1],strides=[1, 6, 6, 1], padding='SAME')# 加入BN函数
def batch_norm_layer(value, train=None, name='batch_norm'):if train is not None:return batch_norm(value, decay=0.9, updates_collections=None, is_training=True)else:return batch_norm(value, decay=0.9, updates_collections=None, is_training=False)# tf Graph Input
x = tf.placeholder(tf.float32, [None, 24, 24, 3])  # cifar data image of shape 24*24*3
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classes
train = tf.placeholder(tf.float32)W_conv1 = weight_variable([5, 5, 3, 64])
b_conv1 = bias_variable([64])x_image = tf.reshape(x, [-1, 24, 24, 3])# 在第一层h_conv1和第二层h_conv2的输出之间卷积之后加入BN层。
h_conv1 = tf.nn.relu(batch_norm_layer((conv2d(x_image, W_conv1) + b_conv1), train))
h_pool1 = max_pool_2x2(h_conv1)W_conv2 = weight_variable([5, 5, 64, 64])
b_conv2 = bias_variable([64])h_conv2 = tf.nn.relu(batch_norm_layer((conv2d(h_pool1, W_conv2) + b_conv2), train))
h_pool2 = max_pool_2x2(h_conv2)W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)nt_hpool3 = avg_pool_6x6(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)# 将原来的学习率改成退化学习率，使用0.04的初始值，让其每1000次退化0.9
# 书本中使用的学习率为0.04，但是不行，将学习率改为0.005后正常。
cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))global_step = tf.Variable(0, trainable=False)
decaylearning_rate = tf.train.exponential_decay(0.005, global_step, 1000, 0.9)train_step = tf.train.AdamOptimizer(decaylearning_rate).minimize(cross_entropy, global_step=global_step)correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))sess = tf.Session()
sess.run(tf.global_variables_initializer())
tf.train.start_queue_runners(sess=sess)
for i in range(1000): # 20000image_batch, label_batch = sess.run([images_train, labels_train])label_b = np.eye(10, dtype=float)[label_batch]  # one hottrain_step.run(feed_dict={x: image_batch, y: label_b, train: 1}, session=sess)if i % 100 == 0:train_accuracy = accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess)print("step %d, training accuracy %g" % (i, train_accuracy))image_batch, label_batch = sess.run([images_test, labels_test])
label_b = np.eye(10, dtype=float)[label_batch]  # one hot
print("finished！ test accuracy %g" % accuracy.eval(feed_dict={x: image_batch, y: label_b}, session=sess))

结果：

step 0, training accuracy 0.226562
step 100, training accuracy 0.484375
step 200, training accuracy 0.492188
step 300, training accuracy 0.445312
step 400, training accuracy 0.515625
step 500, training accuracy 0.515625
step 600, training accuracy 0.554688
step 700, training accuracy 0.53125
step 800, training accuracy 0.617188
step 900, training accuracy 0.578125
finished！ test accuracy 0.53125

第8章 卷积神经网络相关推荐

1. 第五章 卷积神经网络（CNN）

   文章目录 5.1 卷积神经网络的组成层 5.2 卷积如何检测边缘信息? 5.3 卷积层中的几个基本参数? 5.3.1 卷积核大小 5.3.2 卷积核的步长 5.3.3 边缘填充 5.3.4 输入和输出 ...

2. 《Scikit-Learn与TensorFlow机器学习实用指南》第13章 卷积神经网络

   第13章 卷积神经网络 来源:ApacheCN<Sklearn 与 TensorFlow 机器学习实用指南>翻译项目 译者:@akonwang @WilsonQu 校对: @飞龙 ​尽管 ...

3. 第3.1章 卷积神经网络(CNN)——Conv、Pool、FC、Activation Function、BN各个层的作用及原理

   第3.1章 卷积神经网络CNN-不同层的作用 一.Convolution(CONV) 二.Pooling(POOL) 三.Fully Connected(FC) 四.Activation Functi ...

4. 深度学习实战 第6章卷积神经网络笔记

   第6章 卷积神经网络 **卷积神经网络(Convolutional Neural Network,CNN)**是在实际应用中最为成功的一种神经网络,其专门用于处理格状结构数据,比如图片数据就可以看成是 ...

5. 第11章 卷积神经网络（CNNs）

   第11章 卷积神经网络(CNNs) 我们回顾了整个机器学习和深度学习知识,现在我们学习CNNs(Convolutional Neural Networks)以及它在深度学习中的作用.在传统的前馈神经网 ...

6. 曹健老师 TensorFlow2.1 —— 第五章 卷积神经网络

   第一章 第二章 第三章 第四章 本章目的:用图卷积神经网络实现离散数据的分类 ( 以图像分类为例 ) . 5.1 卷积计算过程 在实际项目中,输入神经网络的是具有更高分辨率的彩色图片,使得送入全连接网 ...

7. 第十二章 卷积神经网络实战--猫狗识别

   1.介绍 我们已经学习了如何用传统的神经网络进行机器学习,在本章我们学习一下如何使用简单的神经网络进行图像分类.数据集用的是Kaggle的猫狗数据集.这里只有前100张,如果需要更多的可以去Kaggl ...

8. 第七章 卷积神经网络2(代码实现)

   文章目录 7.1卷积层和池化层实现 7.1.1 4维数组 7.1.2基于im2col的展开 7.1.3卷积层的实现 7.1.4池化层的是实现 7.2CNN实现 7.2.1目录结构如下: 7.2.2结果 ...

9. 深度学习-第二章 卷积神经网络面试题（大厂必问，历经半年整理）

   文章目录 老铁们✌,重要通知

10. CNN的Python实现——第四章：卷积神经网络的结构

    文章目录 第4章 卷积神经网络的结构 4.1 概述 4.1.1 局部连接 4.1.2 参数共享 4.1.3 3D特征图 4.2 卷积层 4.2.1 卷积运算及代码实现 4.2.2 卷积层及代码初级实现 ...

最新文章

1. 中美科技成果转化比较分析
2. 反射,Expression Tree,IL Emit 属性操作对比
3. 如何零基础或者转行数据分析师？
4. Python实用小技能，一个比一个高级！
5. 实验5: IOS的升级与恢复
6. centos挂载windows共享目录
7. 【ES6】对象、函数、数组的扩展
8. 数据库及中间件术语解释
9. .NET : 如何理解字符串和它的字节表现形式
10. DEDE_5.7星星评分插件首发！
11. Java线程池 面试考点
12. Arduino与SG90舵机握手
13. [C#复习向整合]反射 -Assembly与Activator
14. 域名解析、域名转向的作用
15. 集成学习（ensemble learning）基础知识
16. No Matter What
17. python获取文件大小
18. 了解ESP32睡眠模式及其功耗
19. ProFTPD对接LDAP
20. UESTC 1635 最大最小生成树

热门文章

1. javascript 动态画心加文字
2. 解锁计算机桌面,电脑锁屏按什么键解锁
3. svn locked 怎么解决
4. MATLAB RGB转YUV YUV转RGB
5. 第1章第25节：如何通过幻灯片母版统一管理相同类型的幻灯片1 [PowerPoint精美幻灯片实战教程]
6. 动态NAT64实验配置
7. 快乐牛牛终极板creator1.82 shader 挫牌代码
8. 26丨 搜索引擎架构：如何瞬间完成海量数据检索？
9. Sphinx/coreseek/mysql全文检索
10. 前端常见的浏览器兼容性问题及解决方案