(梳理)用Tensorflow实现SE-ResNet(SENet ResNet ResNeXt VGG16)的数据输入,训练,预测的完整代码框架(cifar10准确率90%)

//之前的代码有问题,重发一遍
这里做个整理,打算给一个ResNet的完整代码实现
这篇博客不对ResNet DenseNet思路做详解,只是提供一个大致的代码实现和编写思路
以及一些自己的小实验
当然也包括ResNeXt和SENet
其中SE-ResNeXt是最后一届ImageNet比赛的冠军模型
当然可能我写得烂……参数没精调…准确率提升也不是很明显
首先是数据处理部分,我自己实现过几个框架
多线程数据读入和处理框架(tf.data)
较早版本的队列多线程读入处理框架
最普通的单线程(用法和自带MNIST一样)读取数据
我用了多线程tf.data
Cifar10原生数据集是python打包的
至于图片的话,参考我的这篇,把Cifar10图片提取出来
主要是为了制成TFRecord格式,这样数据读取速度会很快(毕竟自家的东西优化得给足)
VGG AlexNet之前已经跑过了就不跑了,滑动平均下准确率已经在85%以上了
我自己的AlexNet和VGG16
（均嵌入SENet,并且移除全连接层,当然甚至可以把池化也扔掉,用步长2x2的卷积代替池化,这样的话网络看起来像是全卷积网络FCN)
我用全局平均池化+1x1卷积代替全连接层,同时还加上了BN,SENet等后来才有的结构
准确率能跑到85%以上
比较深的网络(VGG19)不加上BN很难收敛,我跑Kaggle的时候跑了10个epoch还是50%的准确率
戳我了解SENet
戳我了解ResNeXt

SENet感觉像是一种Attention机制
ResNet和ResNeXt的差别不是很大,就是卷积分成了几个通路,像Inception那样
不同的是每个通路都是同构的

有三种等价的实现,我这里采用了c的结构
学习率使用warmup策略,因而学习率可以设置到一个比较大的数字(ResNet50设置到了0.036都基本可以正常训练,不过有一定概率中期崩掉)
网络中的一些超参数:
Optimizer: MomentumOptimizer(SGDM)
conv_initializer:xavier(tf.layers.conv2d默认kernel_initializer,实测比stddev=0.01的截断正态分布好)
momentum:0.9
warmup_step：batch_num*10
learning_rate_base：0.012
learning_rate_decay：0.96
learning_rate_step：batch_num
train_epochs：140
batch_size：128
batch_num：50000/batch_size
SE_rate：32
cardinality：8
regularizer_rate：0.008
drop_rate：0.3
exponential_moving_average_decay:0.999
虽然cifar10只有32x32
但是实际输入网络的时候被resize为64x64 或者128x128 (测试准确率差异在1%以内,可以忽略不记)
完整代码:
ResNet50/ResNeXt/SE-AlexNet
数据处理部分：

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from time import time
from PIL import Image
from sklearn.utils import shuffle
from tqdm import tqdm_notebook as tqdm
import os
from tqdm import tqdm_notebook as tqdm
data_path="C:\\Users\\Shijunfeng\\tensorflow_gpu\\dataseats\\cifar10\\Image"
mode_path="C:\\Users\\Shijunfeng\\tensorflow_gpu\\dataseats\\cifar10\\Image\\TFMode"
tf.reset_default_graph()
def imshows(classes,images,labels,index,amount,predictions=None):#classes 类别数组#image 图片数组#labels 标签数组#index amount 从数组第index开始输出amount张照片#prediction 预测结果fig=plt.gcf()fig.set_size_inches(10,20)#大小看怎么调整合适把for i in range(amount):title="lab:"+classes[np.argmax(labels[index+i])]if predictions is not None:title=title+"prd:"+name[np.argmax(predictions[index+i])]ax=plt.subplot(5,6,i+1)#每行五个,输出6行ax.set_title(title)ax.imshow(images[index+i])plt.show()
class DatasetReader(object):def __init__(self,data_path,image_size=None):self.data_path=data_pathself.img_size=image_sizeself.img_size.append(3)self.train_path=os.path.join(data_path,"train")self.test_path=os.path.join(data_path,"test")self.TF_path=os.path.join(data_path,"TFRecordData")self.tf_train_path=os.path.join(self.TF_path,"train")self.tf_test_path=os.path.join(self.TF_path,"test")self.classes=os.listdir(self.train_path)self.__Makedirs()self.train_batch_initializer=Noneself.test_batch_initializer=Noneself.__CreateTFRecord(self.train_path,self.tf_train_path)self.__CreateTFRecord(self.test_path,self.tf_test_path)def __CreateTFRecord(self,read_path,save_path):path=os.path.join(save_path,"data.TFRecord")if os.path.exists(path):print("find file "+(os.path.join(save_path,"data.TFRecords")))returnelse:print("cannot find file %s,ready to recreate"%(os.path.join(save_path,"data.TFRecords")))writer=tf.python_io.TFRecordWriter(path=path)image_path=[]image_label=[]image_size=[int(self.img_size[0]*1.3),int(self.img_size[1]*1.3)]for label,class_name in enumerate(self.classes):class_path=os.path.join(read_path,class_name)for image_name in os.listdir(class_path):image_path.append(os.path.join(class_path,image_name))image_label.append(label)for i in range(5):image_path,image_label=shuffle(image_path,image_label)for i in tqdm(range(len(image_path)),desc="TFRecord"):image,label=Image.open(image_path[i]).resize(image_size,Image.ANTIALIAS),image_label[i]image=image.convert("RGB")image=image.tobytes()example=tf.train.Example(features=tf.train.Features(feature={"label":tf.train.Feature(int64_list=tf.train.Int64List(value=[label])),"image":tf.train.Feature(bytes_list=tf.train.BytesList(value=[image]))}))writer.write(example.SerializeToString())writer.close()def __Makedirs(self):if not os.path.exists(self.TF_path):os.makedirs(self.TF_path)if not os.path.exists(self.tf_train_path):os.makedirs(self.tf_train_path)if not os.path.exists(self.tf_test_path):os.makedirs(self.tf_test_path)def __parsed(self,tensor):raw_image_size=[int(self.img_size[0]*1.3),int(self.img_size[1]*1.3),3]feature=tf.parse_single_example(tensor,features={"image":tf.FixedLenFeature([],tf.string),"label":tf.FixedLenFeature([],tf.int64)})image=tf.decode_raw(feature["image"],tf.uint8)image=tf.reshape(image,raw_image_size)image=tf.random_crop(image,self.img_size)image=tf.image.per_image_standardization(image)label=tf.cast(feature["label"],tf.int32)label=tf.one_hot(label,len(self.classes))return image,labeldef __parsed_distorted(self,tensor):raw_image_size=[int(self.img_size[0]*1.3),int(self.img_size[1]*1.3),3]feature=tf.parse_single_example(tensor,features={"image":tf.FixedLenFeature([],tf.string),"label":tf.FixedLenFeature([],tf.int64)})image=tf.decode_raw(feature["image"],tf.uint8)image=tf.reshape(image,raw_image_size)image=tf.random_crop(image,self.img_size)image=tf.image.random_flip_left_right(image)image=tf.image.random_flip_up_down(image)image=tf.image.random_brightness(image,max_delta=0.4)image=tf.image.random_hue(image,max_delta=0.4)image=tf.image.random_contrast(image,lower=0.7,upper=1.4)image=tf.image.random_saturation(image,lower=0.7,upper=1.4)image=tf.image.per_image_standardization(image)label=tf.cast(feature["label"],tf.int32)label=tf.one_hot(label,len(self.classes))return image,labeldef __GetBatchIterator(self,path,parsed,batch_size):filename=[os.path.join(path,name)for name in os.listdir(path)]dataset=tf.data.TFRecordDataset(filename)dataset=dataset.prefetch(tf.contrib.data.AUTOTUNE)dataset=dataset.apply(tf.data.experimental.shuffle_and_repeat(buffer_size=10000,count=None,seed=99997))dataset=dataset.apply(tf.data.experimental.map_and_batch(parsed,batch_size))dataset=dataset.apply(tf.data.experimental.prefetch_to_device("/gpu:0"))iterator=dataset.make_initializable_iterator()return iterator.initializer,iterator.get_next()def __detail(self,path):Max=-1e9Min=1e9print("train dataset:")path=[os.path.join(path,name)for name in self.classes]for i in range(len(self.classes)):num=len(os.listdir(path[i]))print("%-12s:%3d"%(self.classes[i],num))Max=max(Max,num)Min=min(Min,num)print("max:%d min:%d"%(Max,Min))def detail(self):print("class num:",len(self.classes))self.__detail(self.train_path)self.__detail(self.test_path)def global_variables_initializer(self):initializer=[]initializer.append(self.train_batch_initializer)initializer.append(self.test_batch_initializer)initializer.append(tf.global_variables_initializer())return initializerdef test_batch(self,batch_size):self.test_batch_initializer,batch=self.__GetBatchIterator(self.tf_test_path,self.__parsed,batch_size)return batchdef train_batch(self,batch_size):self.train_batch_initializer,batch=self.__GetBatchIterator(self.tf_train_path,self.__parsed_distorted,batch_size)return batch

网络结构定义:

#tf.reset_default_graph()
def warmup_exponential_decay(warmup_step,rate_base,global_step,decay_step,decay_rate,staircase=False):linear_increase=rate_base*tf.cast(global_step/warmup_step,tf.float32)exponential_decay=tf.train.exponential_decay(rate_base,global_step-warmup_step,decay_step,decay_rate,staircase)return tf.cond(global_step<=warmup_step,lambda:linear_increase,lambda:exponential_decay)class ConvNet(object):def __init__(self,training=False,regularizer_rate=0.0,drop_rate=0.0,SE_rate=32,cardinality=8,average_class=None):self.training=trainingself.SE_rate=SE_rateself.cardinality=cardinalityself.drop_rate=drop_rateif regularizer_rate!=0.0:self.regularizer=tf.contrib.layers.l2_regularizer(regularizer_rate)else:self.regularizer=Noneif average_class is not None:self.ema=average_class.averageelse:self.ema=Nonedef Xavier(self):tf.contrib.layers.xavier_initializer_conv2d()def BatchNorm(self,x,name="BatchNorm"):return tf.layers.batch_normalization(x,training=self.training,reuse=tf.AUTO_REUSE)def Dropout(self,x):return tf.layers.dropout(x,rate=self.drop_rate,training=self.training)def Conv2d(self,x,filters,ksize,strides=[1,1],padding="SAME",activation=tf.nn.relu,use_bn=True,dilation=[1,1],name="conv"):#Conv BN Reluwith tf.variable_scope(name,reuse=tf.AUTO_REUSE):input_channel=x.shape.as_list()[-1]kernel=tf.get_variable("kernel",ksize+[input_channel,filters],initializer=self.Xavier())bias=tf.get_variable("bias",[filters],initializer=tf.constant_initializer(0.0))if self.ema is not None and self.training==False: #滑动平均kernel,bias=self.ema(kernel),self.ema(bias)conv=tf.nn.conv2d(x,kernel,strides=[1]+strides+[1],padding=padding,dilations=[1]+dilation+[1],name=name)bias_add=tf.nn.bias_add(conv,bias)if use_bn:bias_add=self.BatchNorm(bias_add) #BNif activation is not None:bias_add=activation(bias_add) #Reluif self.regularizer is not None:tf.add_to_collection("losses",self.regularizer(kernel))return bias_adddef CardConv2d(self,x,filters,ksize,strides=[1,1],padding="SAME",activation=tf.nn.relu,use_bn=True,dilation=[1,1],name="cardconv"):'''ResNeXt的多路卷积层'''with tf.variable_scope(name,tf.AUTO_REUSE):split=tf.split(x,num_or_size_splits=self.cardinality,axis=-1)filters=int(filters/self.cardinality)for i in range(len(split)):split[i]=self.Conv2d(x,filters,ksize,strides,padding,activation,use_bn,dilation,name)merge=tf.concat(split,axis=-1)return mergedef MaxPool(self,x,ksize,strides=[1,1],padding="SAME",name="max_pool"):input_channel=x.shape.as_list()[-1]return tf.nn.max_pool(x,ksize=[1]+ksize+[1],strides=[1]+strides+[1],padding=padding,name=name)def GlobalAvgPool(self,x,name="GAP"):ksize=[1]+x.shape.as_list()[1:-1]+[1]return tf.nn.avg_pool(x,ksize=ksize,strides=[1,1,1,1],padding="VALID",name=name)def FC_conv(self,x,dim,activation=tf.nn.relu,use_bn=True,name="conv_1x1"):'''1x1卷积代替全连接层'''return self.Conv2d(x,dim,[1,1],activation=activation,use_bn=use_bn,name=name)def SENet(self,x,name="SENet"):with tf.variable_scope(name,tf.AUTO_REUSE):channel=x.shape.as_list()[-1]squeeze=self.GlobalAvgPool(x,name="squeeze")excitation=self.FC_conv(x,int(channel/self.SE_rate),name="excitation_1")excitation=self.FC_conv(excitation,channel,activation=tf.nn.sigmoid,name="excitation_2")return excitation*xdef Residual(self,x,channels,name="Residual"):with tf.variable_scope(name,tf.AUTO_REUSE):res=self.Conv2d(x,channels[0],[1,1],name="conv_1")res=self.Conv2d(res,channels[1],[3,3],name="conv_2")res=self.Conv2d(res,channels[2],[1,1],activation=None,use_bn=True,name="conv_3")res=self.SENet(res)if x.shape.as_list()[-1]!=channels[2]:x=self.Conv2d(x,channels[2],[1,1],activation=None,use_bn=False,name="conv_linear")return tf.nn.relu(res+x)def ResidualX(self,x,channels,name="Residual"):with tf.variable_scope(name,tf.AUTO_REUSE):res=self.Conv2d(x,channels[0],[1,1],name="conv_1")res=self.CardConv2d(res,channels[1],[3,3],name="conv_2")res=self.Conv2d(res,channels[2],[1,1],activation=None,use_bn=True,name="conv_3")res=self.SENet(res)if x.shape.as_list()[-1]!=channels[2]:x=self.Conv2d(x,channels[2],[1,1],activation=None,use_bn=False,name="conv_linear")return tf.nn.relu(res+x)def ResNet(self,x):x=self.Conv2d(x,64,[7,7],[2,2],name="conv1")x=self.MaxPool(x,[3,3],[2,2],name="pool1")x=self.Residual(x,[64,64,128],name="Residual1_1")x=self.Residual(x,[64,64,128],name="Residual1_2")x=self.Residual(x,[64,64,128],name="Residual1_3")x=self.MaxPool(x,[3,3],[2,2],name="pool2")x=self.Residual(x,[128,128,256],name="Residual2_1")x=self.Residual(x,[128,128,256],name="Residual2_2")x=self.Residual(x,[128,128,256],name="Residual2_3")x=self.Residual(x,[128,128,256],name="Residual2_4")x=self.MaxPool(x,[3,3],[2,2],name="pool2")x=self.Residual(x,[256,256,512],name="Residual3_1")x=self.Residual(x,[256,256,512],name="Residual3_2")x=self.Residual(x,[256,256,512],name="Residual3_3")x=self.Residual(x,[256,256,512],name="Residual3_4")x=self.Residual(x,[256,256,512],name="Residual3_5")x=self.Residual(x,[256,256,512],name="Residual3_6")x=self.MaxPool(x,[3,3],[2,2],name="pool3")x=self.Residual(x,[512,512,1024],name="Residual4_1")x=self.Residual(x,[512,512,1024],name="Residual4_2")x=self.Residual(x,[512,512,1024],name="Residual4_3")x=self.Residual(x,[512,512,1024],name="Residual4_4")x=self.MaxPool(x,[3,3],[2,2],name="pool3")x=self.GlobalAvgPool(x,name="GAP")x=self.Dropout(x)x=self.FC_conv(x,512,name="FC1")x=self.Dropout(x)x=self.FC_conv(x,10,activation=None,use_bn=False,name="FC2")return tf.reshape(x,[-1,10])def ResNeXt(self,x):x=self.Conv2d(x,64,[7,7],[2,2],name="conv1")x=self.MaxPool(x,[3,3],[2,2],name="pool1")x=self.ResidualX(x,[64,64,128],name="Residual1_1")x=self.ResidualX(x,[64,64,128],name="Residual1_2")x=self.ResidualX(x,[64,64,128],name="Residual1_3")x=self.MaxPool(x,[3,3],[2,2],name="pool2")x=self.ResidualX(x,[128,128,256],name="Residual2_1")x=self.ResidualX(x,[128,128,256],name="Residual2_2")x=self.ResidualX(x,[128,128,256],name="Residual2_3")x=self.ResidualX(x,[128,128,256],name="Residual2_4")x=self.MaxPool(x,[3,3],[2,2],name="pool2")x=self.ResidualX(x,[256,256,512],name="Residual3_1")x=self.ResidualX(x,[256,256,512],name="Residual3_2")x=self.ResidualX(x,[256,256,512],name="Residual3_3")x=self.ResidualX(x,[256,256,512],name="Residual3_4")x=self.ResidualX(x,[256,256,512],name="Residual3_5")x=self.ResidualX(x,[256,256,512],name="Residual3_6")x=self.MaxPool(x,[3,3],[2,2],name="pool3")x=self.ResidualX(x,[512,512,1024],name="Residual4_1")x=self.ResidualX(x,[512,512,1024],name="Residual4_2")x=self.ResidualX(x,[512,512,1024],name="Residual4_3")x=self.ResidualX(x,[512,512,1024],name="Residual4_4")x=self.MaxPool(x,[3,3],[2,2],name="pool3")x=self.GlobalAvgPool(x,name="GAP")x=self.Dropout(x)x=self.FC_conv(x,512,name="FC1")x=self.Dropout(x)x=self.FC_conv(x,10,activation=None,use_bn=False,name="FC2")return tf.reshape(x,[-1,10])def VGG19(self,x):x=self.Conv2d(x,64,[3,3],[1,1],name="conv1_1")x=self.Conv2d(x,64,[3,3],[1,1],name="conv1_2")x=self.Maxpool2d(x,[3,3],[2,2],name="pool1")x=self.SENet(x,name="SE_1")x=self.Conv2d(x,128,[3,3],[1,1],name="conv2_1")x=self.Conv2d(x,128,[3,3],[1,1],name="conv2_2")x=self.Maxpool2d(x,[3,3],[2,2],name="pool2")x=self.SENet(x,name="SE_2")x=self.Conv2d(x,256,[3,3],[1,1],name="conv3_1")x=self.Conv2d(x,256,[3,3],[1,1],name="conv3_2")x=self.Conv2d(x,256,[3,3],[1,1],name="conv3_3")x=self.Conv2d(x,256,[3,3],[1,1],name="conv3_4")x=self.Maxpool2d(x,[3,3],[2,2],name="pool3")x=self.SENet(x,name="SE_3")x=self.Conv2d(x,256,[3,3],[1,1],name="conv4_1")x=self.Conv2d(x,256,[3,3],[1,1],name="conv4_2")x=self.Conv2d(x,256,[3,3],[1,1],name="conv4_3")x=self.Conv2d(x,256,[3,3],[1,1],name="conv4_4")x=self.Maxpool2d(x,[3,3],[2,2],name="pool4")x=self.SENet(x,name="SE_4")x=self.Conv2d(x,512,[3,3],[1,1],name="conv5_1")x=self.Conv2d(x,512,[3,3],[1,1],name="conv5_2")x=self.Conv2d(x,512,[3,3],[1,1],name="conv5_3")x=self.Conv2d(x,512,[3,3],[1,1],name="conv5_4")x=self.Maxpool2d(x,[3,3],[2,2],name="pool5")x=self.GlobalAveragPool(x,name="GAP")x=self.FC_conv(x,128,name="FC_1")x=self.Dropout(x)x=self.FC_conv(x,10,activation=None,bn=False,name="FC_2")return tf.reshape(x,[-1,10])def AlexNet(self,x):x=self.Conv2d(x,96,[11,11],[4,4],padding="valid",name="conv_1")x=self.Maxpool2d(x,[3,3],[2,2],padding="valid",name="pool_1")x=self.SENet(x,name="SE_1")x=self.Conv2d(x,256,[5,5],[2,2],name="conv_2")x=self.Maxpool2d(x,[3,3],[2,2],name="pool_2")x=self.SENet(x,name="SE_2")x=self.Conv2d(x,512,[3,3],[1,1],name="conv_3")x=self.SENet(x,name="SE_3")x=self.Conv2d(x,384,[3,3],[1,1],name="conv_4")x=self.SENet(x,name="SE_4")x=self.Conv2d(x,256,[3,3],[1,1],name="conv_5")x=self.Maxpool2d(x,[3,3],[2,2],name="pool_3")x=self.SENet(x,name="SE_5")x=self.GlobalAveragPool(x,name="GAP")x=self.SENet(x,name="SE_6")x=self.FC_conv(x,512,name="FC_1")x=self.Dropout(x)x=self.FC_conv(x,512,name="FC_2")x=self.Dropout(x)x=self.FC_conv(x,10,activation=None,bn=False,name="FC_3")return tf.reshape(x,[-1,10])
x=tf.placeholder(tf.float32,[None,128,128,3])
y=tf.placeholder(tf.float32,[None,10])
is_training=tf.placeholder(tf.bool)
global_step=tf.Variable(0,trainable=False)
train_epochs=160
batch_size=64
batch_num=int(50000/batch_size)
SE_rate=32
cardinality=8
regularizer_rate=0.01
drop_rate=0.5
#数据输入
data=DatasetReader(data_path,[128,128])
train_batch=data.train_batch(batch_size=batch_size)
test_batch=data.test_batch(batch_size=256)
#学习率
warmup_step=batch_num*3
learning_rate_base=0.003
learning_rate_decay=0.97
learning_rate_step=batch_num
learning_rate=exponential_decay_with_warmup(warmup_step,learning_rate_base,global_step,learning_rate_step,learning_rate_decay)
#前向预测
forward=ConvNet(training=is_training,regularizer_rate=regularizer_rate,drop_rate=drop_rate,SE_rate=SE_rate,cardinality=cardinality).ResNeXt(x)
#滑动平均&预测
ema_decay=0.999
ema=tf.train.ExponentialMovingAverage(ema_decay,global_step)
ema_op=ema.apply(tf.trainable_variables())
ema_forward=ConvNet(SE_rate=SE_rate,cardinality=cardinality).ResNeXt(x)
prediction=tf.nn.softmax(ema_forward)
correct_pred=tf.equal(tf.argmax(prediction,1),tf.argmax(y,1))
accuracy=tf.reduce_mean(tf.cast(correct_pred,tf.float32))
#优化器
momentum=0.9
update_ops=tf.get_collection(tf.GraphKeys.UPDATE_OPS)
cross_entropy=tf.losses.softmax_cross_entropy(onehot_labels=y,logits=forward,label_smoothing=0.1)
l2_regularizer_loss=tf.add_n(tf.get_collection("losses"))#l2正则化损失
loss_function=cross_entropy+l2_regularizer_loss
update_ops=tf.get_collection(tf.GraphKeys.UPDATE_OPS)
with tf.control_dependencies(update_ops):optimizer=tf.train.MomentumOptimizer(learning_rate,momentum).minimize(loss_function,global_step=global_step)
#模型保存地址
if not os.path.exists(mode_path):os.makedirs(mode_path)
saver=tf.train.Saver(tf.global_variables())

网络训练：

gpu_options = tf.GPUOptions(allow_growth=True)
loss_list,acc_list=[5.0],[0.0]
step,decay,acc,loss=0,0,0,0
maxacc,minloss=0.5,1e5
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_options)) as sess:print("train start")sess.run(data.global_variables_initializer())for epoch in range(train_epochs):bar=tqdm(range(batch_num),unit=" step",desc="epoch %d:"%(epoch+1))bar.set_postfix({"acc:":acc,"loss:":loss})for batch in bar:images,labels=sess.run(train_batch)sess.run(optimizer,feed_dict={x:images,y:labels,is_training:True})step+=1if step%50==0:decay=min(0.9,(step/50+1)/(step/50+10))images,labels=sess.run(test_batch)acc,loss=sess.run([accuracy,loss_function],feed_dict={x:images,y:labels,is_training:False})acc_list.append(decay*acc_list[-1]+(1-decay)*acc)loss_list.append(decay*loss_list[-1]+(1-decay)*loss)bar.set_postfix({"acc:":acc,"loss:":loss})print("update model with acc:%.3f%%"%(acc_list[-1]*100))  #pred=sess.run(prediction,feed_dict={x:images,is_training:False})#print(pred)

可视化损失和准确率变化曲线

plt.plot(acc_list)
plt.legend(["ResNet"],loc="lower right")
plt.show()
plt.plot(loss_list)
plt.legend(["ResNet"],loc="upper right")
np.save("ResNet_acc.npy",acc_list)
np.save("ResNet_loss.npy",loss_list)

用模型来进行预测:

#raw_image std_image是本地图片,std_image输入网络,raw_image为原图片
image_path="C:\\Users\\Shijunfeng\\tensorflow_gpu\\dataseats\\kaggle\\Img"
image_list=[os.path.join(image_path,name)for name in os.listdir(image_path)]
def imshows(classes,images,labels,index,amount,predictions=None):#显示图片和对应标签fig=plt.gcf()fig.set_size_inches(10,50)#大小看怎么调整合适把for i in range(amount):title="lab:"+classes[np.argmax(labels[index+i])]if predictions is not None:title=title+"prd:"+name[np.argmax(predictions[index+i])]ax=plt.subplot(15,2,i+1)#每行五个,输出6行ax.set_title(title)ax.set_xticks([])ax.set_yticks([])ax.imshow(images[index+i])plt.show()
def decode(image):image=Image.open(image)image=tf.image.resize_images(image,[128,128],method=2)image=tf.image.per_image_standardization(image)return image
raw_images=[Image.open(image)for image in image_list]
std_images=[decode(image)for image in image_list]
gpu_options = tf.GPUOptions(allow_growth=True)
with tf.Session(config=tf.ConfigProto(gpu_options=gpu_options)) as sess: #加载模型,这里网络定义部分同样需要sess.run(data.global_variables_initializer())ckp_state=tf.train.get_checkpoint_state(mode_path)if ckp_state and ckp_state.model_checkpoint_path:saver.restore(sess,ckp_state.model_checkpoint_path)images=sess.run(std_image)labels=sess.run(prediction,feed_dict={x:images,is_training:False})imshows(data.classes,images,labels,0,10) #输出前10张照片以及对应标签

最终测试结果：

在测试集上准确率接近100%(后面过拟合了,目前位置没有解决过拟合问题)
在训练集上能达到90%以上的准确率
总共训练耗时81分钟左右
ResNet101
疯狂堆残差快

  def ResNet(self,x):x=self.Conv2d(x,64,[7,7],[2,2],name="conv1")x=self.MaxPool(x,[3,3],[2,2],name="pool1")x=self.Residual(x,[64,64,256],name="Residual1_1")x=self.Residual(x,[64,64,256],name="Residual1_2")x=self.Residual(x,[64,64,256],name="Residual1_3")x=self.MaxPool(x,[3,3],[2,2],name="pool2")x=self.Residual(x,[128,128,512],name="Residual2_1")x=self.Residual(x,[128,128,512],name="Residual2_2")x=self.Residual(x,[128,128,512],name="Residual2_3")x=self.Residual(x,[128,128,512],name="Residual2_4")x=self.MaxPool(x,[3,3],[2,2],name="pool3")x=self.Residual(x,[256,256,1024],name="Residual3_1")x=self.Residual(x,[256,256,1024],name="Residual3_2")x=self.Residual(x,[256,256,1024],name="Residual3_3")x=self.Residual(x,[256,256,1024],name="Residual3_4")x=self.Residual(x,[256,256,1024],name="Residual3_5")x=self.Residual(x,[256,256,1024],name="Residual3_6")x=self.Residual(x,[256,256,1024],name="Residual3_7")x=self.Residual(x,[256,256,1024],name="Residual3_8")x=self.Residual(x,[256,256,1024],name="Residual3_9")x=self.Residual(x,[256,256,1024],name="Residual3_10")x=self.Residual(x,[256,256,1024],name="Residual3_11")x=self.Residual(x,[256,256,1024],name="Residual3_12")x=self.Residual(x,[256,256,1024],name="Residual3_13")x=self.Residual(x,[256,256,1024],name="Residual3_14")x=self.Residual(x,[256,256,1024],name="Residual3_15")x=self.Residual(x,[256,256,1024],name="Residual3_16")x=self.Residual(x,[256,256,1024],name="Residual3_17")x=self.Residual(x,[256,256,1024],name="Residual3_18")x=self.Residual(x,[256,256,1024],name="Residual3_19")x=self.Residual(x,[256,256,1024],name="Residual3_20")x=self.Residual(x,[256,256,1024],name="Residual3_21")x=self.Residual(x,[256,256,1024],name="Residual3_22")x=self.Residual(x,[256,256,1024],name="Residual3_23")x=self.MaxPool(x,[3,3],[2,2],name="pool4")x=self.Residual(x,[512,512,2048],name="Residual4_1")x=self.Residual(x,[512,512,2048],name="Residual4_2")x=self.Residual(x,[512,512,2048],name="Residual4_3")x=self.Residual(x,[512,512,2048],name="Residual4_4")x=self.MaxPool(x,[3,3],[2,2],name="pool5")x=self.GlobalAvgPool(x,name="GAP")x=self.Dropout(x)x=self.FC_conv(x,784,name="FC1")x=self.Dropout(x)x=self.FC_conv(x,10,activation=None,use_bn=False,name="output")return tf.reshape(x,[-1,10])

可以看到网络测试集准确率能够稳定在90%以上
最后20个epoch求平均值得到准确率为90.945%(91%)
最高准确率为94%(batch_size=256)
SE-ResNet101
残差块后面加上SENet子结构即可

    def Residual(self,x,channels,name="Residual"):with tf.variable_scope(name,tf.AUTO_REUSE):res=self.Conv2d(x,channels[0],[1,1],name="conv_1")res=self.Conv2d(res,channels[1],[3,3],name="conv_2")res=self.Conv2d(res,channels[2],[1,1],activation=None,use_bn=True,name="conv_3")res=self.SENet(res)if x.shape.as_list()[-1]!=channels[2]:x=self.Conv2d(x,channels[2],[1,1],activation=None,use_bn=False,name="conv_linear")return tf.nn.relu(res+x)

结果暂时未出,待更新

(梳理)用Tensorflow实现SE-ResNet(SENet ResNet ResNeXt VGG16)的数据输入,训练,预测的完整代码框架(cifar10准确率90%)相关推荐

TensorFlow学习笔记（十一）读取自己的数据进行训练
1. 线性关系数据csv文件读取 x,y 1,2 4,5 6,11 3,6 4,7 5,12 7,13 10,21 11,23 24,50 45,89 50,101 55,111 60,123 70 ...
【图像分类案例】(1) ResNeXt 交通标志四分类，附Tensorflow完整代码
各位同学好,今天和大家分享一下如何使用 Tensorflow 构建 ResNeXt 神经网络模型,通过案例实战 ResNeXt 的训练以及预测过程.每个小节的末尾有网络.训练.预测的完整代码.想要数据 ...
[深度学习-实践]Tensorflow 2.x应用ResNet SeNet网络训练cifar10数据集的模型在测试集上准确率 86%-87%，含完整代码
环境 tensorflow 2.1 最好用GPU Cifar10数据集 CIFAR-10 数据集的分类是机器学习中一个公开的基准测试问题.任务的目标对一组32x32 RGB的图像进行分类,这个数据集涵 ...
[深度学习-TF2实践]应用Tensorflow2.x训练ResNet,SeNet和Inception模型在cifar10，测试集上准确率88.6%
环境 tensorflow 2.1 最好用GPU Cifar10数据集 CIFAR-10 数据集的分类是机器学习中一个公开的基准测试问题.任务的目标对一组32x32 RGB的图像进行分类,这个数据集涵 ...
TensorFlow笔记（9） ResNet
TensorFlow笔记(9) ResNet 1. 残差网络分类问题 2. 数据读取 3. 构建模型 4. 训练模型 5. 评估模型 6. 模型预测 1. 残差网络分类问题非常深的神经网络是很难训练 ...
DL：关于深度学习常用数据集中训练好的权重文件(Deeplab v3、MobileNet、InceptionV3、VGG系列、ResNet、Mask R-CNN )下载地址集合(持续更新)
DL:关于深度学习常用数据集中训练好的权重文件(Deeplab v3.MobileNet.InceptionV3.VGG系列.ResNet.Mask R-CNN )下载地址集合(持续更新) 目录基于 ...
DL之ResNet：ResNet算法的简介(论文介绍)、架构详解、案例应用等配图集合之详细攻略
DL之ResNet:ResNet算法的简介(论文介绍).架构详解.案例应用等配图集合之详细攻略目录 ResNet算法的简介 1.比赛结果-ResNets @ ILSVRC & COCO 20 ...
Caffe 议事（一）：从零开始搭建 ResNet 之残差网络结构介绍和数据准备
声明:Caffe 系列文章是我们实验室黄佳斌大神所写的内部学习文档,已经获得他的授权允许. 本参考资料是在 Ubuntu14.04 版本下进行,并且默认 Caffe 所需的环境已经配置好,下面教大 ...
【神经网络】(17) EfficientNet 代码复现，网络解析，附Tensorflow完整代码
各位同学好,今天和大家分享一下如何使用 Tensorflow 复现 EfficientNet 卷积神经网络模型. EfficientNet 的网络结构和 MobileNetV3 比较相似,建议大家在学 ...
【神经网络】(16) MobileNetV3 代码复现，网络解析，附Tensorflow完整代码
各位同学好,今天和大家分享一下如何使用 Tensorflow 构建 MobileNetV3 轻量化网络模型. MobileNetV3 做了如下改动(1)更新了V2中的逆转残差结构:(2)使用NAS搜索 ...

(梳理)用Tensorflow实现SE-ResNet(SENet ResNet ResNeXt VGG16)的数据输入,训练,预测的完整代码框架(cifar10准确率90%)

(梳理)用Tensorflow实现SE-ResNet(SENet ResNet ResNeXt VGG16)的数据输入,训练,预测的完整代码框架(cifar10准确率90%)相关推荐

最新文章

热门文章