猫狗识别

数据集下载：

　　网盘链接：https://pan.baidu.com/s/1SlNAPf3NbgPyf93XluM7Fg

　　提取密码：hpn4

1. 要导入的包

import os
import time
import numpy as npimport torch
import torch.nn as nn
import torch.nn.functional as F
from torch.utils.data import DataLoader
from torch.utils import data
from torchvision import transforms as T
from PIL import Image

import

2. 模型配置

###################################
# SETTINGS
###################################class Config(object):batch_size = 32max_epoch = 30num_workers = 2lr = 0.001lr_decay = 0.95weight_decay = 0.0001train_data_root = '/home/dong/Documents/DATASET/train'test_data_root = '/home/dong/Documents/DATASET/test'load_dict_path = Noneopt = Config()

SETTINGS

3. 选择DEVICE

device = 'cuda:0' if torch.cuda.is_available() else 'cpu'

4. 数据集

###################################
# DATASETS
###################################class DogCatDataset(data.Dataset):def __init__(self, root, transforms=None, train=True, test=False):super(DogCatDataset, self).__init__()imgs = [os.path.join(root, img) for img in os.listdir(root)]np.random.seed(10000)np.random.permutation(imgs)len_imgs = len(imgs)self.test = test# -----------------------------------------------------------------------------------------# 因为在猫狗数据集中，只有训练集和测试集，但是我们还需要验证集，因此从原始训练集中分离出30%的数据# 用作验证集。# ------------------------------------------------------------------------------------------if self.test:self.imgs = imgselif train:self.imgs = imgs[: int(0.7*len_imgs)]else:self.imgs = imgs[int(0.7*len_imgs): ]if transforms is None:normalize = T.Normalize(mean=[0.485, 0.456, 0.406],std=[0.229, 0.224, 0.225])if self.test or not train:self.transforms = T.Compose([T.Scale(224),T.CenterCrop(224),T.ToTensor(),normalize])else:self.transforms = T.Compose([T.Scale(246),T.RandomCrop(224),T.RandomHorizontalFlip(),T.ToTensor(),normalize])def __getitem__(self, index):# 当前要获取图像的路径img_path = self.imgs[index]if self.test:img_label = int(img_path.split('.')[-2].split('/')[-1])else:img_label = 1 if 'dog' in img_path.split('/')[-1] else 0img_data = Image.open(img_path)img_data = self.transforms(img_data)return img_data, img_labeldef __len__(self):return len(self.imgs)train_dataset = DogCatDataset(root=opt.train_data_root, train=True)   # train=True, test=False -> 训练集
val_dataset = DogCatDataset(root=opt.train_data_root, train=False)    # train=False, test=False -> 验证集
test_dataset = DogCatDataset(root=opt.test_data_root, test=True)      # test=True -> 测试集

train_dataloader = DataLoader(dataset=train_dataset,shuffle=True,batch_size=opt.batch_size,num_workers = opt.num_workers)
val_dataloader = DataLoader(dataset=val_dataset,shuffle=False,batch_size=opt.batch_size,num_workers = opt.num_workers)
test_dataloader = DataLoader(dataset=test_dataset,shuffle=False,batch_size=opt.batch_size,num_workers = opt.num_workers)

DATASETS

5. 检查数据集的 shape

# ------------------------------------------------
#  CHECKING THE DATASETS
# ------------------------------------------------
print("Training set:")
for  images, labels in train_dataloader:print('Image Batch Dimensions:', images.size())print('Label Batch Dimensions:', labels.size())breakprint("Validation set:")
for  images, labels in val_dataloader:print('Image Batch Dimensions:', images.size())print('Label Batch Dimensions:', labels.size())breakprint("Testing set:")
for  images, labels in test_dataloader:print('Image Batch Dimensions:', images.size())print('Label Batch Dimensions:', labels.size())break

View Code

eg:

Training set:
Image Batch Dimensions: torch.Size([32, 3, 224, 224])
Label Batch Dimensions: torch.Size([32])
Validation set:
Image Batch Dimensions: torch.Size([32, 3, 224, 224])
Label Batch Dimensions: torch.Size([32])
Testing set:
Image Batch Dimensions: torch.Size([32, 3, 224, 224])
Label Batch Dimensions: torch.Size([32])

6. 模型定义

###################################################
# MODEL
###################################################class AlexNet(nn.Module):def __init__(self, num_classes=2):       # num_classes代表数据集的类别数
        super(AlexNet, self).__init__()self.features = nn.Sequential(nn.Conv2d(3, 64, kernel_size=(11, 11), stride=(4, 4), padding=2),nn.ReLU(inplace=True),nn.MaxPool2d(kernel_size=(3, 3), stride=2),nn.Conv2d(64, 192, kernel_size=5, padding=2),nn.ReLU(inplace=True),nn.MaxPool2d(kernel_size=3, stride=2),nn.Conv2d(192, 384, kernel_size=3, padding=1),nn.ReLU(inplace=True),nn.Conv2d(384, 256, kernel_size=3, padding=1),nn.ReLU(inplace=True),nn.Conv2d(256, 256, kernel_size=3, padding=1),nn.ReLU(inplace=True),nn.MaxPool2d(kernel_size=3, stride=2))self.avgpool = nn.AdaptiveAvgPool2d((6, 6))self.classifers = nn.Sequential(nn.Dropout(),nn.Linear(256 * 6 * 6, 4096),nn.ReLU(inplace=True),nn.Dropout(),nn.Linear(4096, 4096),nn.ReLU(inplace=True),nn.Linear(4096, num_classes),)def forward(self, x):x = self.features(x)x = self.avgpool(x)x = x.view(x.size(0), 256*6*6)logits = self.classifers(x)probas = F.softmax(logits, dim=1)return logits, probas  # 记载模型def load(self, model_path):self.load_state_dict(torch.load(model_path))# 保存模型def save(self, model_name):# 状态字典的保存格式：文件名 + 日期时间 .pthprefix = 'checkpoints/' + model_name + '_'name = time.strftime(prefix + '%m%d_%H:%M:%S.pth')torch.save(self.state_dict, name)model = AlexNet()
model = model.to(device)

Model

7. 定义优化器

##############################################
# Optimizer
############################################### optimizer = torch.optim.Adam(model.parameters(), lr=opt.lr, weight_decay=opt.weight_decay)

optimizer = torch.optim.SGD(model.parameters(), lr=opt.lr, momentum=0.8)

Optimizer

8. 计算准确率

# -------------------------------------------
# 计算准确率
# -------------------------------------------
def compute_acc(model, dataloader, device):correct_pred, num_examples = 0, 0            # correct_pred 统计正确预测的样本数，num_examples 统计样本总数for i, (features, targets) in enumerate(dataloader):features = features.to(device)targets = targets.to(device)logits, probas = model(features)_, predicted_labels = torch.max(probas, 1)num_examples += targets.size(0)assert predicted_labels.size() == targets.size()correct_pred += (predicted_labels == targets).sum()return correct_pred.float() / num_examples * 100

compute_acc

9. 训练 and 验证

##############################################
# TRAINING and VALIDATION
##############################################

cost_list = []
train_acc_list, val_acc_list = [], []start_time = time.time()for epoch in range(opt.max_epoch):model.train()for batch_idx, (features, targets) in enumerate(train_dataloader):features = features.to(device)targets = targets.to(device)optimizer.zero_grad()logits, probas = model(features)# print(targets.size(), logits.size(), probas.size())cost = F.cross_entropy(logits, targets)# cost = torch.nn.CrossEntropyLoss(logits, targets)
        cost.backward()optimizer.step()cost_list.append(cost.item())if not batch_idx % 50:print('Epoch: %03d/%03d | Batch %03d/%03d | Cost: %.4f'%(epoch+1, opt.max_epoch, batch_idx, len(train_dataloader), cost))model.eval()with torch.set_grad_enabled(False):     # save memory during inference
        train_acc = compute_acc(model, train_dataloader, device=device)val_acc = compute_acc(model, val_dataloader, device=device)print('Epoch: %03d/%03d | Training ACC: %.4f%% | Validation ACC: %.4f%%'%(epoch+1, opt.max_epoch, train_acc, val_acc))train_acc_list.append(train_acc)val_acc_list.append(val_acc)print('Time Elapsed: %.2f min' % ((time.time() - start_time)/60))print('Total Time Elapsed: %.2f min' % ((time.time() - start_time)/60))

Training and Validation

eg:

Epoch: 001/030 | Batch 000/547 | Cost: 0.6945
Epoch: 001/030 | Batch 050/547 | Cost: 0.6920
Epoch: 001/030 | Batch 100/547 | Cost: 0.6942
Epoch: 001/030 | Batch 150/547 | Cost: 0.6926
Epoch: 001/030 | Batch 200/547 | Cost: 0.6926
Epoch: 001/030 | Batch 250/547 | Cost: 0.6946
Epoch: 001/030 | Batch 300/547 | Cost: 0.6920
Epoch: 001/030 | Batch 350/547 | Cost: 0.6951
Epoch: 001/030 | Batch 400/547 | Cost: 0.6943
Epoch: 001/030 | Batch 450/547 | Cost: 0.6946
Epoch: 001/030 | Batch 500/547 | Cost: 0.6932
Epoch: 001/030 | Training ACC: 51.7657% | Validation ACC: 50.8933%
Time Elapsed: 2.98 min
Epoch: 002/030 | Batch 000/547 | Cost: 0.6926
Epoch: 002/030 | Batch 050/547 | Cost: 0.6931
Epoch: 002/030 | Batch 100/547 | Cost: 0.6915
Epoch: 002/030 | Batch 150/547 | Cost: 0.6913
Epoch: 002/030 | Batch 200/547 | Cost: 0.6908
Epoch: 002/030 | Batch 250/547 | Cost: 0.6964
Epoch: 002/030 | Batch 300/547 | Cost: 0.6939
Epoch: 002/030 | Batch 350/547 | Cost: 0.6914
Epoch: 002/030 | Batch 400/547 | Cost: 0.6941
Epoch: 002/030 | Batch 450/547 | Cost: 0.6937
Epoch: 002/030 | Batch 500/547 | Cost: 0.6948
Epoch: 002/030 | Training ACC: 53.0400% | Validation ACC: 52.2933%
Time Elapsed: 6.00 min
...
Epoch: 030/030 | Batch 000/547 | Cost: 0.1297
Epoch: 030/030 | Batch 050/547 | Cost: 0.2972
Epoch: 030/030 | Batch 100/547 | Cost: 0.2468
Epoch: 030/030 | Batch 150/547 | Cost: 0.1685
Epoch: 030/030 | Batch 200/547 | Cost: 0.3452
Epoch: 030/030 | Batch 250/547 | Cost: 0.3029
Epoch: 030/030 | Batch 300/547 | Cost: 0.2975
Epoch: 030/030 | Batch 350/547 | Cost: 0.2125
Epoch: 030/030 | Batch 400/547 | Cost: 0.2317
Epoch: 030/030 | Batch 450/547 | Cost: 0.2464
Epoch: 030/030 | Batch 500/547 | Cost: 0.2487
Epoch: 030/030 | Training ACC: 89.5314% | Validation ACC: 88.6400%
Time Elapsed: 92.85 min
Total Time Elapsed: 92.85 min

View Code

10. 可视化 Loss

plt.plot(cost_list, label='Minibatch cost')
plt.plot(np.convolve(cost_list, np.ones(200,)/200, mode='valid'), label='Running average')
plt.ylabel('Cross Entropy')
plt.xlabel('Iteration')
plt.legend()
plt.show()

visualize loss

eg:

11. 可视化 准确率

plt.plot(np.arange(1, opt.max_epoch+1), train_acc_list, label='Training')
plt.plot(np.arange(1, opt.max_epoch+1), val_acc_list, label='Validation')plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

View Code

eg:

转载于:https://www.cnblogs.com/xxxxxxxxx/p/11525583.html

猫狗识别——PyTorch相关推荐

  1. 详解pytorch实现猫狗识别98%附代码

    详解pytorch实现猫狗识别98%附代码 前言 一.为什么选用pytorch这个框架? 二.实现效果 三.神经网络从头到尾 1.来源:仿照人为处理图片的流程,模拟人们的神经元处理信息的方式 2.总览 ...

  2. PyTorch实现猫狗识别VGG

    转载:https://magolor.cn/2020/01/14/2020-01-14-blog-01/ Kaggle 猫狗识别比赛地址:https://www.kaggle.com/c/dogs-v ...

  3. 基于卷积神经网络的猫狗识别

    卷积神经网络的猫狗识别 任务需求 环境配置 识别猫狗实例 首先导入库 数据预处理: 读取数据: 定义网络: 调整机器学习率 定义训练过程 训练开始 训练结果 用于测试代码 结果 任务需求 按照 htt ...

  4. 使用卷积神经网络处理猫狗识别数据集_v1

    说明 采用pytorch框架实现猫狗识别. 数据集下载 猫狗识别数据集下载: 链接:https://pan.baidu.com/s/1hfzSacJbNBUhcDDtPnzlsg  提取码:fu74 ...

  5. Tensorflow实现kaggle猫狗识别（循序渐进进行网络设计）

    这篇是tensorflow版本,pytorch版本会在下一篇博客给出 友情提示:尽量上GPU,博主CPU上跑一个VGG16花了1.5h... Tensorflow实现kaggle猫狗识别 数据集获取 ...

  6. 基于卷积神经网络(CNN)的猫狗识别

    目录 引言 1.什么是卷积神经网络? 1.1什么是神经网络? 1.2什么是卷积? 2.准备工作 2.1一些知识: 2.2keras 2.3Conv2D 2.4 MaxPooling2D 3.基于卷积神 ...

  7. TensorFlow 卷积神经网络之猫狗识别（二）

    本文是TensorFlow 卷积神经网络之猫狗识别 的姊妹篇,是加载上一篇博客训练好的模型,进行猫狗识别 本文逻辑: 我从网上下载了十几张猫和狗的图片,用于检验我们训练好的模型. 处理我们下载的图片 ...

  8. 毕设：基于CNN卷积神经网络的猫狗识别、狗品种识别(Tensorflow、Keras、Kaggle竞赛)

    基于卷积神经网络的图像识别算法及其应用研究 毕业快一年了,拿出来分享给大家,我和网上唯一的区别就是,我能够同时实现两个方案(猫狗识别和狗品种识别),我当时也是网上各种查,花了2,3个月的时间,一个萝卜 ...

  9. 【深度学习】猫狗识别TensorFlow2实验报告

    实验二:猫狗识别 一.实验目的 利用深度学习实现猫狗动物识别,采用Kaggle提供的公开数据集,训练深度学习模型,对测试集猫狗中的图片准确分类.通过该实验掌握深度学习中基本的CV处理过程. 二.实验原 ...

最新文章

  1. 《Effective C#》某些地方实在是对不起Effective这个词（I）
  2. 5.【练习题】构造方法与重载
  3. 解决嵌入式开发过程中开发板与虚拟机的网络通信问题
  4. MySQL中varchar最大长度是多少
  5. mysql表的增删查改
  6. iOS应用支持IPV6，就那点事儿
  7. Jensen不等式证明
  8. sht11的linux程序,SHT11温湿度传感器的proteus仿真与程序源码
  9. Win10更新失败的解决办法
  10. Zend_Cache
  11. Mongodb节点同步失败状态“ RECOVERING ”恢复
  12. Ubuntu系统下打开chm文件
  13. 南方航空java面试_面试经验 南航面试经历分享
  14. 浅谈压缩感知（二）：理论基础
  15. 新团队成员之间破冰训练
  16. python中日期格式转换，计算单位天数时间差
  17. matlab 调整矩阵形状,在MATLAB中调整3D矩阵(图像)的大小
  18. 阿里云能做棋牌服务器吗？
  19. 计算机二级java真题 百度云,计算机二级Java试题及答案
  20. LeetCode 每日一题 2021-4-25 （递增顺序搜索树）

热门文章

  1. Vue CLI 3 多页应用项目的搭建
  2. AsyncLocal 与 async await
  3. postman接口测试系列：接口参数化和参数的传递
  4. ConVirt 2.0.1中文汉化版
  5. （转）SpringMVC学习(三)——SpringMVC的配置文件
  6. 使用secureCRT连接VMware-Ubuntukylin虚拟机
  7. AlamofireJsonToObjects+EVReflection:一步解决JSON解析
  8. Ajax单元测试傻瓜教程
  9. Spring笔记——2.使用Spring容器
  10. [置顶]       强大的jquery选择器