PyTorch学习—23.PyTorch的基本使用

文章目录

一、 Basic Grammar and common functions
- 1.Tensors and relation to numpy
- 2. Tensor.view( )
- 3.Tensor.squeeze( ) / Tensor.unsqueeze( )
- 4.Variable( 过去式）
- 5.CUDA SEMANTICS
- 6.Gradient
- 7.Using Gradient
二、torch.nn.Module
- 1.nn.Linear( )
- 2.Activation Function
- 3. nn.Conv2d
三、Sequential
- 1.模型参数
- 2.损失函数
四、torch.optim
五、MINST数据集实战

一、 Basic Grammar and common functions

导入包

import torch
import torch.nn as nn
import torch.nn.functional as F
from  torch.autograd import Variable ## Variable概念在新版本已融入Tensor
import numpy as npfrom mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt

1.Tensors and relation to numpy

# 创建np array 和 创建 tensor 的方法 类似
x_numpy = np.array([0.1, 0.2, 0.3])
x_torch = torch.tensor([0.1, 0.2, 0.3])
print('x_numpy, x_torch')
print(x_numpy, x_torch)
print()# Tensor 和 numpy 的转换
print('to and from numpy and pytorch')
print(torch.from_numpy(x_numpy), x_torch.numpy())
print()# numpy 和 Tensor都可以做 +-*%
y_numpy = np.array([3,4,5.])
y_torch = torch.tensor([3,4,5.])
print("x+y")
print(x_numpy + y_numpy, x_torch + y_torch)
print()# np 中的方法， pytorch中很多都有. 例如norm ,二阶范数
# 但有部分高级的矩阵计算Tensor并不支持，比如计算特征值特征向量等。
# 因此numpy还是有存在的必要的。
print("norm")
print(np.linalg.norm(x_numpy), torch.norm(x_torch))
print()# to apply an operation along a dimension,
# we use the dim keyword argument instead of axis
print("mean along the 0th dimension")
x_numpy = np.array([[1,2],[3,4.]])
x_torch = torch.tensor([[1,2],[3,4.]])
print(np.mean(x_numpy, axis=0), torch.mean(x_torch, dim=0)) # dim 3 : [Batch, length, hidden_size]

x_numpy, x_torch
[0.1 0.2 0.3] tensor([0.1000, 0.2000, 0.3000])to and from numpy and pytorch
tensor([0.1000, 0.2000, 0.3000], dtype=torch.float64) [0.1 0.2 0.3]x+y
[3.1 4.2 5.3] tensor([3.1000, 4.2000, 5.3000])norm
0.37416573867739417 tensor(0.3742)mean along the 0th dimension
[2. 3.] tensor([2., 3.])

2. Tensor.view( )

Reshape the tensor which similar to numpy.shape()

N, C, W, H = 10000, 3, 28, 28
X = torch.randn((N, C, W, H))Y =torch.randn((1,1,1))
# print(Y.shape)print(28*28)
# print(X.shape)
print(X.view(N, C, 784).shape) # [10000,3,28,28]  -> [10000,3,784]
print(X.view(-1, C, 392).shape) # automatically choose the 0th dimension

784
torch.Size([10000, 3, 784])
torch.Size([20000, 3, 392])

3.Tensor.squeeze( ) / Tensor.unsqueeze( )

维度的变化（增维或者减维）

print(X.size())
X = X.unsqueeze(0)
print(X.size())
X = X.squeeze(0)
print(X.size())

torch.Size([10000, 3, 28, 28])
torch.Size([1, 10000, 3, 28, 28])
torch.Size([10000, 3, 28, 28])

4.Variable( 过去式）

Tensor是Pytorch中非常高效数据格式，但用曾经的 tensor构建神经网络还远远不够，为了构建计算图，所以Variable（以前的数据形式）是不可或缺的数据形式。Variable是对tensor的封装。 Variable有三个属性：

.data：tensor本身
.grad：对应tensor的梯度
.grad_fn：该Variable是通过什么方式获得的

c = Variable(torch.tensor(2.0),requires_grad =True)
a = torch.tensor(2.0, requires_grad=True) # we set requires_grad=True to let PyTorch know to keep the graph
b = torch.tensor(1.0, requires_grad=True)
c = a + b
d = b + 1
e = c * d
print('c', c)
print('d', d)
print('e', e)

c tensor(3., grad_fn=<AddBackward0>)
d tensor(2., grad_fn=<AddBackward0>)
e tensor(6., grad_fn=<MulBackward0>)

5.CUDA SEMANTICS

cpu = torch.device("cpu")
gpu = torch.device("cuda") # ...我的电脑没装cudax = torch.rand(10)
print(x)
x = x.to(gpu)
print(x)
x = x.to(cpu)
print(x)

tensor([0.1245, 0.2713, 0.5397, 0.3965, 0.9127, 0.1660, 0.2510, 0.9544, 0.5412,0.8402])
tensor([0.1245, 0.2713, 0.5397, 0.3965, 0.9127, 0.1660, 0.2510, 0.9544, 0.5412,0.8402], device='cuda:0')
tensor([0.1245, 0.2713, 0.5397, 0.3965, 0.9127, 0.1660, 0.2510, 0.9544, 0.5412,0.8402])

6.Gradient

def f(x):return (x-2)**2def fp(x):return 2*(x-2)x = torch.tensor([1.0], requires_grad=True)y = f(x)
print(y)
y.backward()print('Analytical f\'(x):', fp(x))
print('PyTorch\'s f\'(x):', x.grad)

tensor([1.], grad_fn=<PowBackward0>)
Analytical f'(x): tensor([-2.], grad_fn=<MulBackward0>)
PyTorch's f'(x): tensor([-2.])

7.Using Gradient

使用梯度反向传播进行梯度下降

x = torch.tensor([5.0], requires_grad=True)
step_size = 0.25print(x.data)
print(x)print('iter,\tx,\tf(x),\tf\'(x),\tf\'(x) pytorch')
for i in range(15):y = f(x) # loss functiony.backward() # compute the gradientprint('{},\t{:.3f},\t{:.3f},\t{:.3f},\t{:.3f}'.format(i, x.item(), f(x).item(), fp(x).item(), x.grad.item()))x = x.data - step_size * x.grad # perform a GD update step# We need to zero the grad variable since the backward()# call accumulates the gradients in .grad instead of overwriting.# The detach_() is for efficiency. You do not need to worry too much about it.x.grad.detach_()# 不清零会造成梯度累加x.grad.zero_()

当我们训练网络的时候可能希望保持一部分的网络参数不变，只对其中一部分的参数进行调整；或者值训练部分分支网络，并不让其梯度对主网络的梯度造成影响。这时候我们就需要使用detach()函数来切断一些分支的反向传播。

tensor([5.])
tensor([5.], requires_grad=True)
iter,   x,  f(x),   f'(x), f'(x) pytorch
0,  5.000,  9.000,  6.000,  6.000

二、torch.nn.Module

Module是PyTorch对张量进行运算的方法。模块被实现为torch.nn.Module类的子类。所有模块都是可调用的，可以组合在一起创建复杂的函数。

1.nn.Linear( )

d_in = 3
d_out = 4
linear_module = nn.Linear(d_in, d_out)  # 3*4 example_tensor = torch.tensor([[1,2,3], [4,5,6]]).float() # 2*3
# applys a linear transformation to the data  2*3 3*4 ..-> 2*4
transformed = linear_module(example_tensor)
print('example_tensor', example_tensor.shape)
print('transormed', transformed.shape)
print()
print('We can see that the weights exist in the background\n')
print('W:', linear_module.weight)
print('b:', linear_module.bias)

example_tensor torch.Size([2, 3])
transormed torch.Size([2, 4])We can see that the weights exist in the backgroundW: Parameter containing:
tensor([[-0.2013, -0.2006,  0.4400],[-0.5354,  0.0908, -0.0148],[-0.5479, -0.3067,  0.1226],[-0.3431, -0.3440, -0.2491]], requires_grad=True)
b: Parameter containing:
tensor([ 0.5456,  0.1424, -0.4834,  0.1194], requires_grad=True)

2.Activation Function

Relu, Tanh, Sigmoid

activation_fn = nn.ReLU() # we instantiate an instance of the ReLU module
example_tensor = torch.tensor([-1.0, 1.0, 0.0])
activated = activation_fn(example_tensor)
print('example_tensor', example_tensor)
print('activated', activated)

example_tensor tensor([-1.,  1.,  0.])
activated tensor([0., 1., 0.])

activation_fn = nn.Tanh() # we instantiate an instance of the Tanh module
example_tensor = torch.tensor([-1.0, 1.0, 0.0])
activated = activation_fn(example_tensor)
print('example_tensor', example_tensor)
print('activated', activated)

example_tensor tensor([-1.,  1.,  0.])
activated tensor([-0.7616,  0.7616,  0.0000])

activation_fn = nn.Sigmoid() # we instantiate an instance of the sigmoid module
example_tensor = torch.tensor([-1.0, 1.0, 0.0])
activated = activation_fn(example_tensor)
print('example_tensor', example_tensor)
print('activated', activated)

example_tensor tensor([-1.,  1.,  0.])
activated tensor([0.2689, 0.7311, 0.5000])

3. nn.Conv2d

torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros')

它期望输入具有特定的维度（

PyTorch学习—23.PyTorch的基本使用相关推荐

Pytorch学习 - Task5 PyTorch卷积层原理和使用
Pytorch学习 - Task5 PyTorch卷积层原理和使用 1. 卷积层 (1)介绍 (torch.nn下的) 1) class torch.nn.Conv1d() 一维卷积层 2) clas ...
Pytorch学习 - Task6 PyTorch常见的损失函数和优化器使用
Pytorch学习 - Task6 PyTorch常见的损失函数和优化器使用官方参考链接 1. 损失函数 (1)BCELoss 二分类计算公式小例子: (2) BCEWithLogitsLoss ...
PyTorch学习记录——PyTorch进阶训练技巧
PyTorch学习记录--PyTorch进阶训练技巧 1.自定义损失函数 1.1 以函数的方式定义损失函数 1.2 以类的方式定义损失函数 1.3 比较与思考 2.动态调整学习率 2.1 官方提供的s ...
PyTorch学习记录——PyTorch生态
Pytorch的强大并不仅局限于自身的易用性,更在于开源社区围绕PyTorch所产生的一系列工具包(一般是Python package)和程序,这些优秀的工具包极大地方便了PyTorch在特定领域的使 ...
1.pytorch学习:安装pytorch
目录安装pytorch 检查pytorch安装是否成功总结安装pytorch 官方网址: Start Locally | PyTorchhttps://pytorch.org/get-start ...
PyTorch学习笔记——pytorch图像处理（transforms）
原始图像 2.图像处理.转不同格式显示 import torch import torchvision import torchvision.transforms as transforms impo ...
add函数 pytorch_Pytorch学习记录-Pytorch可视化使用tensorboardX
Pytorch学习记录-Pytorch可视化使用tensorboardX 在很早很早以前(至少一个半月),我做过几节关于tensorboard的学习记录. https://www.jianshu.co ...
Pytorch学习笔记总结
往期Pytorch学习笔记总结: 1.Pytorch实战笔记_GoAI的博客-CSDN博客 2.Pytorch入门教程_GoAI的博客-CSDN博客 Pytorch系列目录: PyTorch学习笔记( ...
pytorch 学习： STGCN
1 main.ipynb 1.1 导入库 import random import torch import numpy as np import pandas as pd from sklearn. ...
很火的深度学习框架PyTorch怎么用？手把手带你安装配置
导读:本文主要介绍PyTorch的一些基础且常用的概念和模块,以及: 为何选择PyTorch. PyTorch环境的安装与配置. 作者:吴茂贵郁明敏杨本法李涛张粤磊来源:大数据DT(ID:b ...