在Paddle中利用AlexNet测试CIFAR10数据集合
简 介: 利用Paddle框架搭建了AlexNet网络,并在AI Studio上利用其至尊版本测试了AlexNet对于Cifar10的分类效果。 基础的训练在测试集合上的分类效果没有能够超过60%,这对于一些文章中提到的高达80% 的分类效果还有一定的距离。
关键词
: Cifar10,Alexnet
Contents
§01 AlexNet
1.1 背景介绍
在 2021年人工神经网络第四次作业要求 给出了NN课程中的第四次作业要求。关于Cifar10数据集合,在 2021年人工神经网络第四次作业 - 第三题Cifar10 中尝试使用BP,LeNet结构进行训练,在测试集合上的准确性始终无法突破30%。但是测试集合的精度很快就打到的饱和。
在其中简单修改了网络结构,调整学习速率以及使用Dropout层,对于结果影响不带。
参考博文 深度学习识别CIFAR10:pytorch训练LeNet、AlexNet、VGG19实现及比较(二) 中介绍的 AlexNet 的实现方法,在Paddle平台上完成该网络的搭建与测试。
1.2 原文代码
原文根据AlexNet的结构,结合 The CIFAR-10 dataset 图片的特点(32×32×3),对AlexNet网络结构进行了微调:
AlexNet的网络结构:
▲ 图1.2.1 AlexNet的网络结构
对CIFAR10,图片是3232,尺寸远小于227227,因此对网络结构和参数需做微调:
- 卷积层1:核大小7*7,步长2,填充2
- 最后一个max-pool层删除
1.2.1 网络代码
网络定义代码如下:
1 class AlexNet(nn.Module):2 def __init__(self):3 super(AlexNet, self).__init__()4 5 self.cnn = nn.Sequential(6 # 卷积层1,3通道输入,96个卷积核,核大小7*7,步长2,填充27 # 经过该层图像大小变为32-7+2*2 / 2 +1,15*158 # 经3*3最大池化,2步长,图像变为15-3 / 2 + 1, 7*79 nn.Conv2d(3, 96, 7, 2, 2),
10 nn.ReLU(inplace=True),
11 nn.MaxPool2d(3, 2, 0),
12
13 # 卷积层2,96输入通道,256个卷积核,核大小5*5,步长1,填充2
14 # 经过该层图像变为7-5+2*2 / 1 + 1,7*7
15 # 经3*3最大池化,2步长,图像变为7-3 / 2 + 1, 3*3
16 nn.Conv2d(96, 256, 5, 1, 2),
17 nn.ReLU(inplace=True),
18 nn.MaxPool2d(3, 2, 0),
19
20 # 卷积层3,256输入通道,384个卷积核,核大小3*3,步长1,填充1
21 # 经过该层图像变为3-3+2*1 / 1 + 1,3*3
22 nn.Conv2d(256, 384, 3, 1, 1),
23 nn.ReLU(inplace=True),
24
25 # 卷积层3,384输入通道,384个卷积核,核大小3*3,步长1,填充1
26 # 经过该层图像变为3-3+2*1 / 1 + 1,3*3
27 nn.Conv2d(384, 384, 3, 1, 1),
28 nn.ReLU(inplace=True),
29
30 # 卷积层3,384输入通道,256个卷积核,核大小3*3,步长1,填充1
31 # 经过该层图像变为3-3+2*1 / 1 + 1,3*3
32 nn.Conv2d(384, 256, 3, 1, 1),
33 nn.ReLU(inplace=True)
34 )
35
36 self.fc = nn.Sequential(
37 # 256个feature,每个feature 3*3
38 nn.Linear(256*3*3, 1024),
39 nn.ReLU(),
40 nn.Linear(1024, 512),
41 nn.ReLU(),
42 nn.Linear(512, 10)
43 )
44
45 def forward(self, x):
46 x = self.cnn(x)
47
48 # x.size()[0]: batch size
49 x = x.view(x.size()[0], -1)
50 x = self.fc(x)
51
52 return x
1.3 Paddle模型实现
利用Paddle中的神经网络模型构建Alexnet。
1.3.1 搭建Alexnet网络
(1)网络代码
import paddleclass alexnet(paddle.nn.Layer):def __init__(self, ):super(alexnet, self).__init__()self.conv1 = paddle.nn.Conv2D(in_channels=3, out_channels=96, kernel_size=7, stride=2, padding=2)self.conv2 = paddle.nn.Conv2D(in_channels=96, out_channels=256, kernel_size=5, stride=1, padding=2)self.conv3 = paddle.nn.Conv2D(in_channels=256, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv4 = paddle.nn.Conv2D(in_channels=384, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv5 = paddle.nn.Conv2D(in_channels=384, out_channels=256, kernel_size=3, stride=1, padding=1)self.mp1 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.mp2 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.L1 = paddle.nn.Linear(in_features=256*3*3, out_features=1024)self.L2 = paddle.nn.Linear(in_features=1024, out_features=512)self.L3 = paddle.nn.Linear(in_features=512, out_features=10)def forward(self, x):x = self.conv1(x)x = paddle.nn.functional.relu(x)x = self.mp1(x)x = self.conv2(x)x = paddle.nn.functional.relu(x)x = self.mp2(x)x = self.conv3(x)x = paddle.nn.functional.relu(x)x = self.conv4(x)x = paddle.nn.functional.relu(x)x = self.conv5(x)x = paddle.nn.functional.relu(x)x = paddle.flatten(x, start_axis=1, stop_axis=-1)x = self.L1(x)x = paddle.nn.functional.relu(x)x = self.L2(x)x = paddle.nn.functional.relu(x)x = self.L3(x)return x
(2)网络结构
应用paddle.summary检查网络结构是否正确。
model = alexnet()paddle.summary(model, (100,3,32,32))
---------------------------------------------------------------------------Layer (type) Input Shape Output Shape Param #
===========================================================================Conv2D-16 [[100, 3, 32, 32]] [100, 96, 15, 15] 14,208 MaxPool2D-7 [[100, 96, 15, 15]] [100, 96, 7, 7] 0 Conv2D-17 [[100, 96, 7, 7]] [100, 256, 7, 7] 614,656 MaxPool2D-8 [[100, 256, 7, 7]] [100, 256, 3, 3] 0 Conv2D-18 [[100, 256, 3, 3]] [100, 384, 3, 3] 885,120 Conv2D-19 [[100, 384, 3, 3]] [100, 384, 3, 3] 1,327,488 Conv2D-20 [[100, 384, 3, 3]] [100, 256, 3, 3] 884,992 Linear-10 [[100, 2304]] [100, 1024] 2,360,320 Linear-11 [[100, 1024]] [100, 512] 524,800 Linear-12 [[100, 512]] [100, 10] 5,130
===========================================================================
Total params: 6,616,714
Trainable params: 6,616,714
Non-trainable params: 0
---------------------------------------------------------------------------
Input size (MB): 1.17
Forward/backward pass size (MB): 39.61
Params size (MB): 25.24
Estimated Total Size (MB): 66.02
---------------------------------------------------------------------------{'total_params': 6616714, 'trainable_params': 6616714}
在网络设计过程中,往往会出现结构性差错的地方就在卷积层与全连接层之间出现,在进行Flatten(扁平化)之后,出现数据维度对不上。可以在网络定义的过程中,首先将Flatten之后的全连接层去掉,通过paddle.summary输出结构确认卷积层数出为 256×3×3之后,再将全连接层接上。如果出现差错,可以进行每一层校验。
1.4 Cifar10训练AlexNet
1.4.1 载入数据
import sys,os,math,time
import matplotlib.pyplot as plt
from numpy import *import paddle
from paddle.vision.transforms import Normalize
normalize = Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5], data_format='HWC')from paddle.vision.datasets import Cifar10
cifar10_train = Cifar10(mode='train', transform=normalize)
cifar10_test = Cifar10(mode='test', transform=normalize)train_dataset = [cifar10_train.data[id][0].reshape(3,32,32) for id in range(len(cifar10_train.data))]
train_labels = [cifar10_train.data[id][1] for id in range(len(cifar10_train.data))]class Dataset(paddle.io.Dataset):def __init__(self, num_samples):super(Dataset, self).__init__()self.num_samples = num_samplesdef __getitem__(self, index):data = train_dataset[index]label = train_labels[index]return paddle.to_tensor(data,dtype='float32'), paddle.to_tensor(label,dtype='int64')def __len__(self):return self.num_samples_dataset = Dataset(len(cifar10_train.data))
train_loader = paddle.io.DataLoader(_dataset, batch_size=100, shuffle=True)
1.4.2 构建网络
class alexnet(paddle.nn.Layer):def __init__(self, ):super(alexnet, self).__init__()self.conv1 = paddle.nn.Conv2D(in_channels=3, out_channels=96, kernel_size=7, stride=2, padding=2)self.conv2 = paddle.nn.Conv2D(in_channels=96, out_channels=256, kernel_size=5, stride=1, padding=2)self.conv3 = paddle.nn.Conv2D(in_channels=256, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv4 = paddle.nn.Conv2D(in_channels=384, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv5 = paddle.nn.Conv2D(in_channels=384, out_channels=256, kernel_size=3, stride=1, padding=1)self.mp1 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.mp2 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.L1 = paddle.nn.Linear(in_features=256*3*3, out_features=1024)self.L2 = paddle.nn.Linear(in_features=1024, out_features=512)self.L3 = paddle.nn.Linear(in_features=512, out_features=10)def forward(self, x):x = self.conv1(x)x = paddle.nn.functional.relu(x)x = self.mp1(x)x = self.conv2(x)x = paddle.nn.functional.relu(x)x = self.mp2(x)x = self.conv3(x)x = paddle.nn.functional.relu(x)x = self.conv4(x)x = paddle.nn.functional.relu(x)x = self.conv5(x)x = paddle.nn.functional.relu(x)x = paddle.flatten(x, start_axis=1, stop_axis=-1)x = self.L1(x)x = paddle.nn.functional.relu(x)x = self.L2(x)x = paddle.nn.functional.relu(x)x = self.L3(x)return xmodel = alexnet()
1.4.3 训练网络
test_dataset = [cifar10_test.data[id][0].reshape(3,32,32) for id in range(len(cifar10_test.data))]
test_label = [cifar10_test.data[id][1] for id in range(len(cifar10_test.data))]test_input = paddle.to_tensor(test_dataset, dtype='float32')
test_l = paddle.to_tensor(array(test_label)[:,newaxis])optimizer = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())
def train(model):model.train()epochs = 2accdim = []lossdim = []testaccdim = []for epoch in range(epochs):for batch, data in enumerate(train_loader()):out = model(data[0])loss = paddle.nn.functional.cross_entropy(out, data[1])acc = paddle.metric.accuracy(out, data[1])loss.backward()optimizer.step()optimizer.clear_grad()accdim.append(acc.numpy())lossdim.append(loss.numpy())predict = model(test_input)testacc = paddle.metric.accuracy(predict, test_l)testaccdim.append(testacc.numpy())if batch%10 == 0 and batch>0:print('Epoch:{}, Batch: {}, Loss:{}, Accuracys:{}{}'.format(epoch, batch, loss.numpy(), acc.numpy(), testacc.numpy()))plt.figure(figsize=(10, 6))plt.plot(accdim, label='Accuracy')plt.plot(testaccdim, label='Test')plt.xlabel('Step')plt.ylabel('Acc')plt.grid(True)plt.legend(loc='upper left')plt.tight_layout()train(model)
1.4.4 训练结果
- 训练参数:
-
BatchSize
:100
LearningRate
:0.001
如果BatchSize过小,训练速度变慢。
▲ 图1.4.1 训练精度和测试精度变化曲线
- 训练参数:
-
BatchSize
:5000
LearningRate
:0.0005
BatchSize:5000,Lr=0.001, DropOut:0.2:
▲ 图1.4.2 训练精度和测试精度的变化
▲ 图1.4.3 训练精度和测试精度的变化
BatchSize:5000,Lr=0.0001, DropOut:0.2:
▲ 图1.4.4 BatchSize:5000,Lr=0.0001, DropOut:0.2
▲ 图A1.4.2 BatchSize:5000,Lr=0.001, DropOut:0.2
BatchSize:5000,Lr=0.0005, DropOut:0.5:
▲ 图1.4.6 BatchSize:5000,Lr=0.0005, DropOut:0.5
BatchSize:5000,Lr=0.0001, DropOut:0.5:
▲ 图1.4.7 BatchSize:5000,Lr=0.0001, DropOut:0.5
※ 总 结 ※
利用Paddle框架搭建了AlexNet网络,并在AI Studio上利用其至尊版本测试了AlexNet对于Cifar10的分类效果。 基础的训练在测试集合上的分类效果没有能够超过60%,这对于一些文章中提到的高达80% 的分类效果还有一定的距离。
■ 相关文献链接:
- 2021年人工神经网络第四次作业要求
- 2021年人工神经网络第四次作业 - 第三题Cifar10
- 深度学习识别CIFAR10:pytorch训练LeNet、AlexNet、VGG19实现及比较(二)
- AlexNet
- The CIFAR-10 dataset
● 相关图表链接:
- 图1.2.1 AlexNet的网络结构
- 图1.4.1 训练精度和测试精度变化曲线
- 图1.4.2 训练精度和测试精度的变化
- 图1.4.3 训练精度和测试精度的变化
#!/usr/local/bin/python
# -*- coding: gbk -*-
#============================================================
# TESTALEXNET.PY -- by Dr. ZhuoQing 2021-12-20
#
# Note:
#============================================================from headm import * # =import paddle
from paddle.vision.transforms import Normalize
normalize = Normalize(mean=[0.5,0.5,0.5], std=[0.5,0.5,0.5], data_format='HWC')from paddle.vision.datasets import Cifar10
cifar10_train = Cifar10(mode='train', transform=normalize)
cifar10_test = Cifar10(mode='test', transform=normalize)#------------------------------------------------------------
train_dataset = [cifar10_train.data[id][0].reshape(3,32,32) for id in range(len(cifar10_train.data))]
train_labels = [cifar10_train.data[id][1] for id in range(len(cifar10_train.data))]#------------------------------------------------------------
class Dataset(paddle.io.Dataset):def __init__(self, num_samples):super(Dataset, self).__init__()self.num_samples = num_samplesdef __getitem__(self, index):data = train_dataset[index]label = train_labels[index]return paddle.to_tensor(data,dtype='float32'), paddle.to_tensor(label,dtype='int64')def __len__(self):return self.num_samples_dataset = Dataset(len(cifar10_train.data))
train_loader = paddle.io.DataLoader(_dataset, batch_size=5000, shuffle=True)#------------------------------------------------------------
class alexnet(paddle.nn.Layer):def __init__(self, ):super(alexnet, self).__init__()self.conv1 = paddle.nn.Conv2D(in_channels=3, out_channels=96, kernel_size=7, stride=2, padding=2)self.conv2 = paddle.nn.Conv2D(in_channels=96, out_channels=256, kernel_size=5, stride=1, padding=2)self.conv3 = paddle.nn.Conv2D(in_channels=256, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv4 = paddle.nn.Conv2D(in_channels=384, out_channels=384, kernel_size=3, stride=1, padding=1)self.conv5 = paddle.nn.Conv2D(in_channels=384, out_channels=256, kernel_size=3, stride=1, padding=1)self.mp1 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.mp2 = paddle.nn.MaxPool2D(kernel_size=3, stride=2)self.L1 = paddle.nn.Linear(in_features=256*3*3, out_features=1024)self.L2 = paddle.nn.Linear(in_features=1024, out_features=512)self.L3 = paddle.nn.Linear(in_features=512, out_features=10)def forward(self, x):x = self.conv1(x)x = paddle.nn.functional.relu(x)x = self.mp1(x)x = self.conv2(x)x = paddle.nn.functional.relu(x)x = self.mp2(x)x = self.conv3(x)x = paddle.nn.functional.relu(x)x = self.conv4(x)x = paddle.nn.functional.relu(x)x = self.conv5(x)x = paddle.nn.functional.relu(x)x = paddle.flatten(x, start_axis=1, stop_axis=-1)x = self.L1(x)x = paddle.nn.functional.relu(x)x = self.L2(x)x = paddle.nn.functional.relu(x)x = self.L3(x)return xmodel = alexnet()#------------------------------------------------------------
test_dataset = [cifar10_test.data[id][0].reshape(3,32,32) for id in range(len(cifar10_test.data))]
test_label = [cifar10_test.data[id][1] for id in range(len(cifar10_test.data))]test_input = paddle.to_tensor(test_dataset, dtype='float32')
test_l = paddle.to_tensor(array(test_label)[:,newaxis])#------------------------------------------------------------
optimizer = paddle.optimizer.Adam(learning_rate=0.001, parameters=model.parameters())
def train(model):model.train()epochs = 20accdim = []lossdim = []testaccdim = []for epoch in range(epochs):for batch, data in enumerate(train_loader()):out = model(data[0])loss = paddle.nn.functional.cross_entropy(out, data[1])acc = paddle.metric.accuracy(out, data[1])loss.backward()optimizer.step()optimizer.clear_grad()accdim.append(acc.numpy())lossdim.append(loss.numpy())predict = model(test_input)testacc = paddle.metric.accuracy(predict, test_l)testaccdim.append(testacc.numpy())if batch%5 == 0 and batch>0:print('Epoch:{}, Batch: {}, Loss:{}, Accuracys:{}{}'.format(epoch, batch, loss.numpy(), acc.numpy(), testacc.numpy()))
# if epoch%2 == 0 and epoch>0:
# print('Epoch:{}, Loss:{}, Accuracys:{}{}'.format(epoch, loss.numpy(), acc.numpy(), testacc.numpy()))plt.figure(figsize=(10, 6))plt.plot(accdim, label='Accuracy')plt.plot(testaccdim, label='Test')plt.xlabel('Step')plt.ylabel('Acc')plt.grid(True)plt.legend(loc='upper left')plt.tight_layout()train(model)#------------------------------------------------------------paddle.save(model.state_dict(), './work/cifar10_alexnet.pdparams')#------------------------------------------------------------paddle.summary(model,(100, 3,32,32))#------------------------------------------------------------predict = model(paddle.to_tensor(test_dataset, dtype='float32'))test_target = paddle.fluid.layers.argmax(predict, axis=1).numpy()
printt(test_target)
origin_label = array(test_label)errorid = where(test_target != origin_label)[0]
printt(errorid, len(errorid))#------------------------------------------------------------cifarname = {0:'airplane', 1:'automobile', 2:'bird', 3:'cat',4:'deear', 5:'dog', 6:'frog', 7:'horse', 8:'ship', 9:'truck'}PIC_ROW = 4
PIC_COL = 6
plt.figure(figsize=(12,8))
for j in range(PIC_ROW):for i in range(PIC_COL):id = i+j*PIC_COLplt.subplot(PIC_ROW, PIC_COL, id+1)plt.axis('off')eid = errorid[id]tid = test_target[eid]plt.imshow(test_dataset[eid].swapaxes(1,2).T, cmap=plt.cm.gray)plt.title(cifarname[tid], fontsize=12, color='blue')#------------------------------------------------------------
# END OF FILE : TESTALEXNET.PY
#============================================================
在Paddle中利用AlexNet测试CIFAR10数据集合相关推荐
- TF之DCGAN:基于TF利用DCGAN测试MNIST数据集并进行生成过程全记录
TF之DCGAN:基于TF利用DCGAN测试MNIST数据集并进行生成 目录 测试结果 测试过程全记录 测试结果 train_00_0099 train_00_0799 train_00_0899 t ...
- MVC中利用ViewBag传递Json数据时的前端处理方法
** MVC中利用ViewBag传递Json数据时的前端处理方法 ** 用viewBag传递Json字符串到前端时,json字符串中的"会被转义为& quot,前端处理方法为@Htm ...
- 【Python基础】在pandas中利用hdf5高效存储数据
1 简介 HDF5(Hierarchical Data Formal)是用于存储大规模数值数据的较为理想的存储格式. 其文件后缀名为h5,存储读取速度非常快,且可在文件内部按照明确的层次存储数据,同一 ...
- Python应用实战-在pandas中利用hdf5高效存储数据
1 简介 HDF5(Hierarchical Data Formal)是用于存储大规模数值数据的较为理想的存储格式. 其文件后缀名为h5,存储读取速度非常快,且可在文件内部按照明确的层次存储数据,同一 ...
- oracle索引使用例子,Oracle中利用函数索引处理数据倾斜案例
关于B-Tree.Bitmap.函数索引的相关内容请参考另一篇博文: Oracle中B-Tree.Bitmap和函数索引使用案例总结 通常来说,索引选取的数据列最好为分散度高.选择性好.从索引树结构的 ...
- C#中利用事件来传递数据(EventArgs)
前言 前文介绍在GUI编程中的事件,其中说到EventHandler委托类型在声明时,必须有两个参数变量,第一个是object类型,第二个是EventArgs类型. 对于EventArgs类型,有2个 ...
- ado.net mysql 批量插入_[Dapper].NET/C#程序开发中使用Dapper批量插入数据集合的方法应该如何实现?...
问题描述 使用Dapper如何向数据库中批量插入数据或者说使用Dapper如何插入一个集合(List),在没有使用Dapper(使用Ado.net)的情况下,我们通常的实现方式如下: try { co ...
- C语言中利用联合体巧妙处理数据
在C语言处理数据的时候,经常会遇到将一个整形数据拆分成高位和低位,或者是将两个字符型数据组合成一个整形数据.通常的做法是将数据左移或者右移对数据进行组合或者拆分.现在可以利用联合体的特性来处理这一类数 ...
- SQLServer中利用NTILE函数对数据进行分组的一点使用
本文出处:http://www.cnblogs.com/wy123/p/6908377.html NTILE函数可以按照指定的排序规则,对数据按照指定的组数(M个对象,按照某种排序分N个组)进行分组, ...
最新文章
- 500个普通人名_2020年世界500强汽车行业排名:大众公司第一,丰田汽车公司第二...
- hadoop2.6.5安装文档及解决root用户无法ssh localhost的问题
- vue tag=“li“ 和event原有样式渲染不出来
- Linux CENTOS6.5 图形命令互转
- jzoj6065-[NOI2019模拟2019.3.18]One?One!【FFT】
- MyEclipse中SVN的常见的使用方法
- 【软件工程】软件开发的本质
- 3.7.2 - Escape Sequences
- 小例子:在 ECharts 中实现日历图
- left join, right join , inner join, join, union的意义
- vim中编辑了代码 但是提示can not write的解决办法和代码对齐办法
- 我与希赛PMP的不解之缘
- windows11条件下将yafu路径添加到环境变量
- C语言实现HDB3编码与译码
- MRI_Made_Easy 磁共振成像原理-物理基础5
- 为什么qq相册显示服务器数据错误,玩qq空间常遇到的问题及解决办法_qq空间
- win10摄像头打开后黑屏怎么回事?(驱动重新装了、注册表按照网上的方法也改过了、相机隐私设置也打开了,总之各种方法都尝试了还是打开黑屏)
- 机顶盒装linux教程,一种Linux机顶盒焦点控制方法与流程
- 台式电脑一般价钱多少_一般普通台式电脑价格多少为好?
- 停止员工拖延症!工时管理系统的作用之一