ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解

环境

tensorflow 2.1
最好用GPU

模型

ResNet

训练数据

Cifar10 或者 Cifar 100

训练集上准确率：92%左右
验证集上准确率：87.6%左右
测试集上准确率：86.38%
训练时间在GPU上：一小时多
权重大小：3.42 MB

原理介绍

ResNet 的核心思想是引入一个所谓的「恒等快捷连接」（identity shortcut connection），直接跳过一个或多个层，如下图所示：

残差块

代码实践

关键代码段

def block1(x, filters, kernel_size=3, stride=1,conv_shortcut=True, name=None):bn_axis = 3# 此代码目的就是保证最后相加的时候最后一个维度相同， 因为Stride不同，ResNet靠的是Stride来降第1，2维度，不是下采样if conv_shortcut is True:shortcut = layers.Conv2D(4 * filters, 1, strides=stride,name=name + '_0_conv')(x)shortcut = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name + '_0_bn')(shortcut)else:shortcut = x# 此代码目的就是保证最后一个维度与shortcut相同x = layers.Conv2D(filters, 1, strides=stride, name=name + '_1_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name + '_1_bn')(x)                 x = layers.Activation('relu', name=name + '_1_relu')(x)# 和一般的卷积一样x = layers.Conv2D(filters, kernel_size, padding='SAME',  name=name + '_2_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5,  name=name + '_2_bn')(x)x = layers.Activation('relu', name=name + '_2_relu')(x)# 保证第1和2的维度和shortcut 相同x = layers.Conv2D(4 * filters, 1, name=name + '_3_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5, name=name + '_3_bn')(x)# ResNet的核心，权重相加x = layers.Add(name=name + '_add')([shortcut, x])x = layers.Activation('relu', name=name + '_out')(x)return x

完整代码

import tensorflow as tf
import tensorflow.keras as keras
import tensorflow.keras.layers as layers
import image_augument.image_augment as image_augment
import time as time
import tensorflow.keras.preprocessing.image as image
import matplotlib.pyplot as plt
import os
from keras_applications import densenetdef block1(x, filters, kernel_size=3, stride=1,conv_shortcut=True, name=None):bn_axis = 3if conv_shortcut is True:shortcut = layers.Conv2D(4 * filters, 1, strides=stride,name=name + '_0_conv')(x)shortcut = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5,name=name + '_0_bn')(shortcut)else:shortcut = xx = layers.Conv2D(filters, 1, strides=stride, name=name + '_1_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5,name=name + '_1_bn')(x)x = layers.Activation('relu', name=name + '_1_relu')(x)x = layers.Conv2D(filters, kernel_size, padding='SAME',name=name + '_2_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5,name=name + '_2_bn')(x)x = layers.Activation('relu', name=name + '_2_relu')(x)x = layers.Conv2D(4 * filters, 1, name=name + '_3_conv')(x)x = layers.BatchNormalization(axis=bn_axis, epsilon=1.001e-5,name=name + '_3_bn')(x)x = layers.Add(name=name + '_add')([shortcut, x])x = layers.Activation('relu', name=name + '_out')(x)return x
def stack1(x, filters, blocks, stride1=2, name=None):x = block1(x, filters, stride=stride1, name=name + '_block1')for i in range(2, blocks + 1):x = block1(x, filters, conv_shortcut=False, name=name + '_block' + str(i))return xdef my_densenet():inputs = keras.Input(shape=(32, 32, 3), name='img')# （none, 32,32, 32）x = stack1(inputs, 8, 3, stride1=1, name='conv2')#（none, 16,16, 64）x = stack1(x, 16, 4, name='conv3')#（none, 8,8, 128）x = stack1(x, 32, 23, name='conv4')#（none, 4,4, 256）x = stack1(x, 64, 3, name='conv5')x = layers.GlobalAveragePooling2D(name='avg_pool')(x)x = layers.Dense(10, activation='softmax', name='fc')(x)model = keras.Model(inputs, x, name='my_ResNet101')return model
def my_model():denseNet = keras.applications.DenseNet121(input_shape=(32,32,3), include_top=True, weights=None, classes=10)denseNet = my_densenet()denseNet.compile(optimizer=keras.optimizers.Adam(),loss=keras.losses.SparseCategoricalCrossentropy(),#metrics=['accuracy'])metrics=[keras.metrics.SparseCategoricalAccuracy()])denseNet.summary()keras.utils.plot_model(denseNet, 'my_ResNet101.png', show_shapes=True)return denseNetcurrent_max_loss = 9999weight_file='./weights6_2/model.h5'
log_file = 'logs6_2'def train_my_model(deep_model):(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()train_datagen = image.ImageDataGenerator(rescale=1 / 255,rotation_range=40,  # 角度值，0-180.表示图像随机旋转的角度范围width_shift_range=0.2,  # 平移比例，下同height_shift_range=0.2,shear_range=0.2,  # 随机错切变换角度zoom_range=0.2,  # 随即缩放比例horizontal_flip=True,  # 随机将一半图像水平翻转fill_mode='nearest'  # 填充新创建像素的方法)test_datagen = image.ImageDataGenerator(rescale=1 / 255)validation_datagen = image.ImageDataGenerator(rescale=1 / 255)train_generator = train_datagen.flow(x_train[:45000], y_train[:45000], batch_size=128)# train_generator = train_datagen.flow(x_train, y_train, batch_size=128)validation_generator = validation_datagen.flow(x_train[45000:], y_train[45000:], batch_size=128)test_generator = test_datagen.flow(x_test, y_test, batch_size=128)begin_time = time.time()if os.path.isfile(weight_file):print('load weight')deep_model.load_weights(weight_file)def save_weight(epoch, logs):global current_max_lossif(logs['val_loss'] is not None and  logs['val_loss']< current_max_loss):current_max_loss = logs['val_loss']print('save_weight', epoch, current_max_loss)deep_model.save_weights(weight_file)batch_print_callback = keras.callbacks.LambdaCallback(on_epoch_end=save_weight)callbacks = [tf.keras.callbacks.EarlyStopping(patience=4, monitor='loss'),batch_print_callback,# keras.callbacks.ModelCheckpoint('./weights/model.h5', save_best_only=True),tf.keras.callbacks.TensorBoard(log_dir=log_file)]print(train_generator[0][0].shape)history = deep_model.fit_generator(train_generator, steps_per_epoch=351, epochs=200, callbacks=callbacks,validation_data=validation_generator, validation_steps=39, initial_epoch = 0)global current_max_lossif (history.history['val_loss'] is not None and history.history['val_loss'] < current_max_loss):current_max_loss = history['val_loss']print('save_weight', current_max_loss)deep_model.save_weights(weight_file)result = deep_model.evaluate_generator(test_generator, verbose=2)print(result)print('time', time.time() - begin_time)def show_result(history):plt.plot(history.history['loss'])plt.plot(history.history['val_loss'])plt.plot(history.history['sparse_categorical_accuracy'])plt.plot(history.history['val_sparse_categorical_accuracy'])plt.legend(['loss', 'val_loss', 'sparse_categorical_accuracy', 'val_sparse_categorical_accuracy'],loc='upper left')plt.show()print(history)show_result(history)
def test_module(deep_model):(x_train, y_train), (x_test, y_test) = tf.keras.datasets.cifar10.load_data()test_datagen = image.ImageDataGenerator(rescale=1 / 255)test_generator = test_datagen.flow(x_test, y_test, batch_size=128)begin_time = time.time()if os.path.isfile(weight_file):print('load weight')deep_model.load_weights(weight_file)result = deep_model.evaluate_generator(test_generator, verbose=2)print(result)print('time', time.time() - begin_time)def predict_module(deep_model):x_train, y_train, x_test, y_test = image_augment.get_all_train_data(False)import numpy as npif os.path.isfile(weight_file):print('load weight')deep_model.load_weights(weight_file)print(y_test[0:20])for i in range(20):img = x_test[i][np.newaxis, :]/255y_ = deep_model.predict(img)v  = np.argmax(y_)print(v, y_test[i])if __name__ == '__main__':#my_densenet()deep_model = my_model()train_my_model(deep_model)#test_module(deep_model)#predict_module(deep_model)

测试集上运行结果

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解相关推荐

Inception网络运行在Cifar10 测试集87.88% Tensorflow 2.1 小白从代码实践中理解
环境 tensorflow 2.1 最好用GPU 模型 Inception 训练数据 Cifar10 或者 Cifar 100 训练集上准确率:93%左右验证集上准确率:88%左右测试集上准确率: ...
filepath直接指定到文件名吗_PyTest运行指定的测试集
帮忙多点点文章末右下角的"好看"支持下,也可以将本文分享到朋友圈或你身边的朋友,谢谢在PyTest中,提供了几种从命令行运行指定的测试集方式. 方式一,文件/模块级运行,即指定运 ...
dropout是什么？为什么dropout管用？测试集上是否需要使用dropout？说明为什么神经网络中的dropout可以作为正则化？
dropout是什么?测试集上是否需要使用dropout?说明为什么神经网络中的dropout可以作为正则化? 目录
机器学习中的训练集验证集测试集的关系
1.划分测试集目的为了了解一个模型对新样本的泛化能力,唯一的办法是:让已经训练好的模型真正的处理新的样本. 解决方法: 将原始数据划分成两个部分:训练集测试集.可以使用训练集来训练模型,然后用测试 ...
1.1 训练/开发/测试集-深度学习第二课《改善深层神经网络》-Stanford吴恩达教授
←上一篇 ↓↑ 下一篇→ 4.9 总结习题 (第一课) 回到目录 1.2 偏差/方差训练/开发/测试集 (Train/Dev/Test Sets) 大家可能已经了解了,那么本周,我们将继续学习如何有 ...
机器学习：样本集、验证集（开发集）、测试集
样本集.验证集(开发集).测试集. Ripley, B.D(1996)在他的经典专著Pattern Recognition and Neural Networks中给出了这三个词的定义. Traini ...
深度学习的实用层面 —— 1.1 训练/开发/测试集
在配置训练.验证和测试数据集的过程中做出正确决策会在很大程度上帮助大家创建高效的神经网络. 在训练神经网络时,我们需要做出很多决策,例如神经网络分多少层,每层含有多少个隐藏单元,学习速率是多少,各层采 ...
在测试集PARSEC测试集简介与使用
随着大数据中海量数据的处理,传统的DRAM作为主存不在适用.市场上衍生出许多主存替代版本,其中对于自己设计的存储方案进行测试时,有部分研究者使用PARSEC作为测试集. PARSEC测试集:共享存储计 ...
NLP文本情感分析:测试集loss比训练集loss大很多，训练集效果好测试集效果差的原因
NLP情感分析:测试集loss比训练集loss大很多一.前言二.原因一.前言最近在学习神经网络自然语言处理的相关知识,发现运行的之后测试集的loss比训练集的loss大很多,而accuracy ...

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解

环境

模型

训练数据

原理介绍

代码实践

完整代码

测试集上运行结果

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解相关推荐

最新文章

热门文章

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中 理解

环境

模型

训练数据

原理介绍

代码实践

完整代码

测试集上运行结果

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中 理解相关推荐

最新文章

热门文章

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解

ResNet 运行在Cifar10 测试集86.38% Tensorflow 2.1 小白从代码实践中理解相关推荐