Neural Networks

使用torch.nn包来构建神经网络。nn包依赖autograd包来定义模型并求导。一个nn.Module包含各个层和一个forward(input)方法，该方法返回output。

它是一个简单的前馈神经网络，它接受一个输入，然后一层接着一层地传递，最后输出计算的结果。

神经网络的典型训练过程如下：

1.定义包含一些可学习的参数(或者叫权重)神经网络模型；
2.在数据集上迭代；
3.通过神经网络处理输入；
4.计算损失(输出结果和正确值的差值大小)；
5.将梯度反向传播回网络的参数；
6.更新网络的参数，主要使用如下简单的更新原则： weight = weight - learning_rate * gradient

定义网络

import torch
import torch.nn as nn
import torch.nn.functional as Fclass Net(nn.Module):def __init__(self):#复制并使用Net的父类的初始化方法，即先运行nn.Module的初始化函数super(Net, self).__init__()# 1 input image channel, 6 output channels, 5x5 square convolution# kernelself.conv1 = nn.Conv2d(1, 6, 5)#self.conv2 = nn.Conv2d(6, 16, 5)# an affine operation: y = Wx + bself.fc1 = nn.Linear(16 * 5 * 5, 120)#16*5*5：一开始的input是32*32，经过两次卷积池化之后的大小就是16（卷积核）*5（宽）*5（高），  #然后再把它通过view后。全部输入到第一个全连接层的时候，input大小就是16*5*5。self.fc2 = nn.Linear(120, 84)self.fc3 = nn.Linear(84, 10)#定义该神经网络的向前传播函数，该函数必须定义，一旦定义成功，向后传播函数也会自动生成（autograd）def forward(self, x):# Max pooling over a (2, 2) window#输入x经过卷积conv1之后，经过激活函数ReLU，使用2x2的窗口进行最大池化Max pooling，然后更新到x。x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2))# If the size is a square you can only specify a single number#输入x经过卷积conv2之后，经过激活函数ReLU，使用2x2的窗口进行最大池化Max pooling，然后更新到x。x = F.max_pool2d(F.relu(self.conv2(x)), 2)# view函数将张量x变形成一维的向量形式，总特征数并不改变，为接下来的全连接作准备 x = x.view(-1, self.num_flat_features(x))# 输入x经过全连接1，再经过ReLU激活函数，然后更新xx = F.relu(self.fc1(x))# 输入x经过全连接2，再经过ReLU激活函数，然后更新xx = F.relu(self.fc2(x))# 输入x经过全连接3，然后更新xx = self.fc3(x)return xdef num_flat_features(self, x):size = x.size()[1:]  # all dimensions except the batch dimensionnum_features = 1for s in size:num_features *= sreturn num_featuresnet = Net()
print(net)

Net((conv1): Conv2d(1, 6, kernel_size=(5, 5), stride=(1, 1))(conv2): Conv2d(6, 16, kernel_size=(5, 5), stride=(1, 1))(fc1): Linear(in_features=400, out_features=120, bias=True)(fc2): Linear(in_features=120, out_features=84, bias=True)(fc3): Linear(in_features=84, out_features=10, bias=True)
)

net.parameters()返回可被学习的参数（权重）列表和值

params = list(net.parameters())
print(len(params))
print(params[0].size())  # conv1's .weight

10
torch.Size([6, 1, 5, 5])

详细查看网络的参数

print(list(net.named_parameters()))

[('conv1.weight', Parameter containing:
tensor([[[[-0.0864,  0.1581,  0.1053, -0.0674,  0.0957],[ 0.0101, -0.1520, -0.0849,  0.1507,  0.1818],[ 0.0768,  0.0109, -0.0293,  0.0672,  0.0463],[-0.1766,  0.0446,  0.1647,  0.1359, -0.1144],[ 0.1169,  0.1585, -0.1872,  0.1876, -0.1431]]],[[[-0.0481, -0.0338, -0.0865,  0.1113,  0.0281],[-0.0110, -0.1859,  0.0683, -0.1943,  0.1785],[-0.0686,  0.0450,  0.1983, -0.0679, -0.1770],[ 0.1880,  0.0102, -0.0149,  0.1218, -0.1315],[-0.1027,  0.0082, -0.0568,  0.1253, -0.0648]]],[[[-0.1574, -0.1198,  0.1627, -0.1704,  0.0758],[ 0.0399,  0.0644, -0.1151, -0.1112,  0.0837],[-0.0485,  0.0777,  0.0932,  0.0524,  0.0491],[-0.0652,  0.0403, -0.0345, -0.1912,  0.0434],[ 0.0401, -0.1806,  0.1038, -0.0035,  0.1026]]],[[[ 0.0366, -0.1110, -0.1358, -0.0388, -0.0136],[ 0.0998,  0.0714,  0.0491, -0.1141, -0.1639],[-0.0606,  0.1912,  0.0887, -0.1987,  0.1686],[-0.0851, -0.0431,  0.0035,  0.0634,  0.1633],[-0.1842, -0.1439, -0.1178,  0.1330,  0.1844]]],[[[ 0.1000,  0.1853,  0.0840,  0.0314,  0.1282],[ 0.0399,  0.1136,  0.1043, -0.1371,  0.0194],[-0.0125, -0.0674,  0.1311,  0.0281, -0.0820],[-0.0661,  0.0128, -0.1913, -0.0224,  0.0260],[ 0.0750,  0.0270,  0.1146,  0.1069,  0.0046]]],[[[ 0.0444,  0.1200, -0.1469,  0.1303, -0.0595],[-0.1066, -0.1726,  0.1489, -0.0327, -0.0576],[ 0.1739,  0.1475,  0.1767,  0.0594,  0.1139],[ 0.0412, -0.1624, -0.1075, -0.1895, -0.1300],[-0.1598, -0.1082, -0.0381, -0.0411,  0.0309]]]], requires_grad=True)), ('conv1.bias', Parameter containing:
tensor([ 0.2000,  0.1894,  0.1787,  0.0921, -0.0161,  0.1778],requires_grad=True)), ('conv2.weight', Parameter containing:
tensor([[[[-4.6751e-02,  6.2027e-02,  6.8025e-02,  6.2829e-02,  5.9207e-02],[ 3.4858e-02, -4.6945e-02,  3.6028e-03,  9.2037e-03, -5.0105e-02],[ 1.9097e-02, -5.0400e-02,  6.0714e-02, -2.4261e-02,  2.4184e-02],[ 6.4232e-03,  4.2253e-02,  3.8566e-02,  7.8425e-02, -4.7642e-02],[-1.6932e-02,  7.0821e-02, -7.0269e-02,  4.5952e-02,  4.7312e-02]],[[ 2.4947e-02, -5.6880e-02, -5.0089e-02, -1.8905e-02, -5.6852e-02],[-2.1401e-02,  7.6649e-02,  5.6511e-02,  2.9109e-02, -4.3187e-02],[-3.8747e-02, -5.5686e-02, -6.8277e-02,  4.3523e-02, -3.1327e-02],[ 4.6242e-02, -6.7128e-02,  2.1462e-02,  2.1463e-02, -5.7259e-02],[ 8.5649e-03,  4.0587e-02,  7.0886e-02,  3.4475e-02,  4.8482e-02]],[[ 3.4679e-02, -5.7377e-02, -3.8839e-02,  5.0153e-02,  4.7679e-02],[-6.6696e-02, -5.2049e-02,  6.1444e-03, -1.0752e-02, -3.8693e-02],[-6.1365e-02,  6.7706e-02, -8.0698e-02, -2.0662e-02, -6.6051e-02],[-6.0435e-02, -4.1243e-02,  7.0561e-02, -7.8558e-02, -7.2256e-02],[-2.9384e-02, -5.9898e-02,  1.4626e-02,  1.2504e-02, -1.6448e-03]],[[-8.3191e-03, -2.4209e-02, -2.4460e-02, -3.7102e-02, -2.8801e-02],[ 7.0775e-02, -7.7456e-02,  1.6669e-02, -5.7365e-02,  5.1116e-02],[-2.2186e-03,  7.1819e-02,  1.4087e-02,  1.2796e-02, -8.0466e-02],[-2.2954e-02,  1.7675e-02, -6.7700e-02,  4.2731e-02, -4.5965e-02],[ 1.9493e-02,  2.8682e-02, -3.5744e-03, -6.2132e-02, -1.6093e-02]],[[-4.6586e-02, -5.1907e-03, -2.0616e-02, -5.0160e-03, -1.0218e-02],[ 3.0281e-02,  1.0599e-02, -4.4635e-02, -6.2386e-03,  1.2839e-02],[ 6.3183e-02, -4.7749e-02, -1.5905e-02,  3.7993e-03,  6.9684e-03],[-4.3931e-02,  1.2768e-02,  8.1540e-02,  4.5033e-02, -4.2173e-02],[ 5.9427e-02, -4.7729e-02,  4.9988e-02,  5.0745e-02,  6.8716e-02]],[[ 3.1921e-02,  7.3147e-02,  8.1235e-02,  4.7650e-02,  1.7855e-02],[ 4.9684e-02,  4.8450e-02,  2.7255e-02, -8.2837e-03, -7.4009e-02],[ 6.2119e-02, -7.7966e-02, -6.6083e-02,  5.2162e-02,  2.8479e-02],[ 7.4365e-02, -2.8491e-02,  9.6219e-03,  1.9529e-02, -2.1051e-03],[ 6.5328e-02,  3.6120e-02, -2.6581e-02,  6.8440e-02, -1.8014e-02]]],[[[ 2.6748e-02,  5.5181e-02,  2.2528e-02,  6.6341e-03,  5.5057e-02],[ 3.3438e-02, -3.9520e-02, -3.8108e-02, -4.8951e-03,  4.3503e-02],[-1.7223e-02, -6.1210e-02, -1.6332e-02,  7.2617e-02,  1.2141e-02],[ 1.5687e-02, -7.5238e-02, -3.2138e-02, -6.2180e-03,  7.4173e-02],[ 1.3187e-02,  3.0043e-02, -4.9262e-02,  1.0382e-02, -5.2707e-02]],[[-6.1639e-02, -1.6759e-03,  9.1899e-03, -2.0154e-02,  4.7649e-02],[ 4.4110e-02, -4.4745e-02, -6.4256e-02, -5.3946e-02,  1.4076e-02],[ 2.5333e-02,  6.0833e-02, -3.7473e-03, -3.7682e-02, -7.7613e-03],[ 1.2864e-02,  6.8993e-02, -2.3292e-02, -3.1719e-02,  1.6382e-02],[ 7.3083e-02,  7.3863e-02, -2.6046e-02, -4.9677e-02,  1.5290e-02]],[[ 7.4857e-02, -5.2538e-02, -8.0590e-02, -6.6272e-02, -6.3306e-02],[-4.3562e-02, -6.7632e-02, -7.2244e-02,  6.6524e-02,  3.8218e-02],[ 2.9765e-02, -1.5771e-02, -1.2642e-02, -6.1716e-02, -6.1412e-02],[-1.2431e-02,  7.5718e-02, -4.0329e-02,  2.4516e-02, -7.6501e-02],[-2.5373e-02, -7.6981e-02,  4.2991e-02, -5.8949e-02, -8.1620e-02]],[[-2.8247e-04, -4.3295e-02, -7.0514e-04, -6.0201e-02, -9.4066e-03],[ 2.6099e-02, -6.8910e-02, -5.6376e-02, -7.7094e-02, -3.9688e-02],[ 2.1004e-02,  3.8341e-02, -4.4436e-03,  4.8700e-02,  1.1442e-02],[-2.7794e-02, -2.9512e-02, -8.3854e-03,  2.5953e-02, -5.8595e-02],[-9.8773e-03,  3.8318e-02, -6.4992e-02, -6.4106e-02,  2.1669e-02]],[[-5.6786e-02, -2.3078e-02, -6.3540e-02,  6.4722e-02, -3.6102e-03],[ 2.6318e-03, -5.8278e-02, -7.0678e-02,  6.4511e-02,  4.9434e-02],[-1.7529e-02, -3.6845e-02, -4.1389e-02, -1.7150e-02, -1.7080e-02],[-5.1267e-02, -5.4333e-02,  3.7604e-02,  2.5120e-02,  4.5091e-03],[ 6.0331e-02,  1.0586e-02, -4.9640e-03, -4.0446e-02, -5.2631e-02]],[[ 4.2650e-02,  8.0488e-02,  2.2548e-02,  2.5217e-02,  3.9701e-02],[ 2.4243e-02,  3.7999e-02, -2.1265e-02, -4.0239e-02,  6.2443e-02],[-6.5399e-02, -4.2270e-03,  7.1457e-02,  7.6897e-02,  7.7068e-03],[ 3.0916e-02, -1.0364e-02,  1.8940e-02, -5.4591e-02,  3.6602e-02],[ 7.2685e-02, -5.0956e-02, -7.2337e-02,  7.0943e-02,  7.7663e-02]]],[[[ 6.8438e-02, -4.1001e-02,  4.9182e-02, -1.0405e-02,  5.2470e-02],[ 5.1253e-03,  3.2332e-02,  5.4736e-03,  7.6816e-02, -2.4995e-02],[-1.5874e-02, -6.0776e-02,  6.4105e-03,  3.5231e-02,  2.4181e-03],[ 4.9819e-03,  8.2966e-03,  4.6681e-02,  6.0710e-02,  2.3426e-02],[ 3.0622e-03, -4.2392e-02,  7.9669e-02, -6.0518e-02, -7.7647e-02]],[[ 5.6804e-02,  3.2624e-02,  2.6365e-02,  1.9988e-02,  5.0879e-02],[ 5.7976e-03,  5.6419e-02, -6.0317e-02, -6.3418e-02,  7.9154e-02],[ 5.4380e-02,  5.0265e-02, -3.4530e-02, -3.4289e-02, -5.7359e-02],[-2.0039e-02, -1.1091e-02,  6.2006e-02,  4.2279e-02,  1.3524e-02],[-5.1808e-02,  1.6028e-02, -1.3753e-02, -4.4715e-02,  6.4370e-02]],[[-7.5229e-02, -3.1394e-02, -5.7542e-02, -2.5208e-03,  3.8752e-02],[-3.8169e-02, -5.5508e-02,  4.4447e-03,  1.3224e-02, -9.8716e-03],[ 5.8168e-02, -3.0163e-02, -1.7469e-02, -6.4066e-02,  5.9589e-02],[ 1.6919e-02,  2.7378e-02,  2.6752e-02, -2.9841e-03,  6.6029e-03],[ 4.5793e-02, -1.0817e-02,  5.7493e-02, -1.9364e-02,  6.9119e-02]],[[-1.7484e-02, -7.0515e-02, -2.1695e-02,  2.5091e-02,  2.9314e-02],[-3.3494e-02,  8.0373e-02, -5.5042e-02,  5.6274e-02, -6.3458e-02],[-7.4188e-02, -7.3826e-02,  6.3572e-03,  7.2682e-02, -4.4463e-02],[-3.3026e-02, -5.7821e-02,  5.5200e-02, -5.6740e-02,  7.5848e-02],[-1.8855e-02,  5.5604e-03,  2.7479e-02,  3.2113e-02, -7.1571e-02]],[[-5.9697e-02,  5.5090e-02,  3.4306e-02, -2.8067e-02,  1.0584e-02],[ 1.9661e-03, -8.2800e-03,  4.1372e-02,  7.3996e-02, -2.4893e-02],[ 5.5144e-02,  4.6595e-03,  5.8500e-02, -5.4630e-02, -4.6803e-04],[-1.2829e-02, -7.8442e-02, -5.3737e-02,  4.0796e-02,  6.9439e-02],[-7.3417e-02,  6.3357e-02,  2.7502e-02, -3.9882e-02,  6.1540e-03]],[[ 3.7075e-02,  6.7935e-02,  2.5044e-02, -4.7587e-02,  4.8540e-02],[ 5.1211e-02, -1.1007e-02,  7.1380e-02,  7.2305e-02, -4.2107e-02],[ 2.6421e-02,  2.0946e-02, -4.6829e-02,  2.6259e-02, -6.5568e-02],[-4.4335e-02, -8.0355e-02, -2.2185e-02,  8.1275e-02, -4.5410e-02],[-7.9133e-02,  7.0272e-03, -5.9176e-02, -7.3026e-02,  6.6735e-02]]],...,[[[ 4.7453e-02,  1.6624e-03, -4.9159e-02, -6.3126e-02,  7.6058e-02],[-7.6433e-02,  6.6995e-02,  7.6168e-02,  2.9466e-02, -4.2630e-02],[-8.1500e-02,  1.6365e-02,  4.9923e-02,  9.0797e-03, -7.3443e-02],[-9.7154e-03, -3.5494e-02,  7.9512e-02, -3.8434e-02, -1.7049e-02],[ 3.3263e-02, -4.0653e-02,  1.1763e-02,  1.1847e-02,  1.3137e-02]],[[-7.0701e-02,  5.2604e-02, -2.0280e-02, -5.1948e-02,  6.1880e-02],[ 5.0646e-02, -6.0460e-02,  2.2518e-02,  5.4762e-02,  2.6427e-04],[ 5.5895e-02, -3.1711e-02, -7.7939e-02, -3.7241e-02,  5.3921e-02],[ 8.0110e-02,  3.4243e-02, -3.8894e-02,  4.3316e-02, -6.2829e-02],[-2.0985e-02,  2.0693e-02, -4.0446e-02,  5.4924e-02, -7.0742e-02]],[[ 5.8691e-02,  5.7382e-02,  6.1324e-03,  7.0543e-02,  8.1322e-02],[ 1.7258e-02, -8.9442e-03,  5.5113e-02, -1.6722e-03, -3.3769e-02],[-1.2999e-02, -3.7392e-02,  5.2130e-02, -6.1703e-02,  2.5096e-02],[ 2.4746e-02,  1.8096e-02, -5.4290e-03,  5.6583e-03, -3.2907e-02],[ 2.8275e-02, -2.2063e-03,  5.2794e-02,  4.5287e-02,  1.2212e-02]],[[-6.6913e-02, -1.5204e-02, -2.6589e-02, -3.9329e-02, -1.7749e-02],[ 5.4253e-02, -4.8495e-02,  1.3793e-02,  6.2208e-02,  3.4392e-02],[-3.9233e-02,  1.6612e-02, -1.9025e-02, -4.1305e-02, -4.1636e-02],[ 6.3880e-03, -7.3584e-02, -7.6999e-02,  3.9100e-02,  6.4777e-02],[ 6.0852e-02, -1.7153e-02,  6.4263e-02, -6.6408e-02, -2.8885e-02]],[[-5.6493e-02, -8.9312e-03, -6.7856e-02, -2.2723e-03,  6.3610e-02],[-8.4754e-03, -4.5080e-02,  2.0656e-02,  7.2127e-02,  6.0980e-02],[-2.8793e-02,  2.5090e-02,  6.1947e-02, -5.9376e-02, -7.8424e-02],[-7.0324e-02, -2.1602e-03, -3.6420e-02,  3.6624e-02,  6.1857e-02],[-1.1167e-02,  8.7337e-04,  1.7752e-02,  7.5435e-02, -2.4711e-02]],[[ 7.4637e-02,  6.9347e-02,  3.9538e-02,  4.4932e-02,  3.9881e-02],[-2.0344e-02, -5.1092e-02,  7.0027e-03, -8.0631e-02, -1.7589e-02],[-6.1046e-02,  3.9349e-02, -7.4361e-02,  6.2805e-02, -6.0173e-02],[ 3.9680e-02,  6.1395e-02,  6.7319e-02,  6.8641e-03, -2.2438e-03],[ 3.9790e-02,  1.5846e-02,  4.8661e-02,  7.7811e-02,  4.8407e-02]]],[[[-5.9373e-03, -1.3359e-02, -4.8370e-02, -4.6881e-02,  6.3805e-02],[ 6.6859e-02,  5.4224e-02, -1.5097e-02,  2.9913e-02,  4.1902e-02],[-4.7653e-02, -5.5400e-02, -4.5789e-02, -2.2652e-02,  1.4833e-03],[ 4.5867e-03,  9.4236e-03,  6.7346e-02, -2.9957e-02,  2.4964e-02],[ 1.4203e-02,  7.3159e-02, -3.3535e-02, -4.0967e-02, -6.9503e-02]],[[ 1.8814e-02,  2.5195e-02,  2.5055e-02, -6.1476e-02,  2.6273e-02],[ 8.0260e-02, -1.8977e-03, -1.4657e-02, -3.2918e-02,  7.8980e-02],[ 2.2730e-02, -5.7422e-02, -5.7983e-02,  7.5397e-02, -5.1492e-02],[-7.7555e-02, -3.0075e-02, -3.7065e-02,  2.9274e-02,  7.8566e-02],[ 4.8337e-02, -8.0466e-02, -3.4578e-02,  7.4706e-02, -4.7404e-02]],[[ 7.3456e-02,  4.5859e-02,  4.8964e-02,  6.4293e-02,  3.2808e-02],[ 8.0656e-02,  8.7235e-03,  5.7811e-02, -7.8568e-02, -2.5927e-02],[-7.4949e-02, -8.0697e-02,  2.6019e-03, -4.1251e-02, -2.1189e-02],[ 3.4552e-02, -6.9636e-02, -2.4231e-02,  7.0092e-03,  1.3193e-03],[-5.3746e-02, -1.5284e-02, -2.3633e-02, -7.1587e-02, -7.4092e-03]],[[-1.0503e-02, -4.2436e-02,  6.0257e-02,  3.7005e-02, -5.7754e-03],[ 5.1638e-02, -2.6558e-02,  1.1289e-02, -6.4803e-02, -3.0699e-02],[ 4.8014e-02, -5.4398e-02,  5.9667e-02,  7.3184e-03, -5.9711e-02],[-7.0265e-02, -5.3415e-02,  7.1987e-02,  8.8195e-03,  3.9838e-03],[ 4.5838e-02,  2.0373e-02,  7.5955e-02, -1.4837e-02,  5.9894e-04]],[[ 2.7225e-03,  8.1387e-02,  7.1768e-02,  5.3041e-02, -1.8938e-02],[-4.4462e-03, -7.4515e-03,  7.2474e-02, -8.1187e-02,  6.8177e-02],[-6.3188e-02,  7.8637e-02,  7.5461e-02, -4.3640e-02, -2.7294e-02],[ 1.4516e-03, -2.1540e-02,  6.3983e-02, -5.5237e-02,  6.4640e-02],[ 6.7225e-02, -3.2269e-02,  5.2291e-02,  4.1978e-02,  1.0545e-02]],[[-3.9995e-02,  1.5795e-02, -1.1020e-02,  3.0129e-02, -7.5448e-02],[ 4.8956e-02,  7.5318e-02,  4.4730e-03, -4.0962e-02,  2.6889e-02],[ 6.6497e-02,  1.6028e-02, -7.3739e-02, -3.7472e-02,  7.3378e-03],[ 6.1110e-02, -7.4697e-02, -3.3396e-02, -5.3662e-02,  1.9632e-02],[-2.7660e-02, -6.3403e-02,  7.5017e-02, -6.2683e-05, -3.7116e-02]]],[[[-1.8004e-02,  6.5819e-02,  6.2223e-02,  1.6332e-02,  1.8143e-02],[-3.6172e-02, -6.9717e-02, -5.4694e-03,  2.3101e-02,  7.9629e-03],[ 6.3475e-02,  3.4132e-02, -5.2767e-03, -3.8902e-02, -5.6896e-02],[ 6.7207e-02, -4.2420e-02,  5.5340e-02,  3.7744e-02,  4.0298e-02],[-6.8564e-02,  2.5046e-02,  6.8845e-03,  4.5561e-02,  6.5920e-02]],[[ 4.8047e-02, -2.8234e-02,  7.3325e-02, -4.8417e-02,  3.9393e-02],[ 6.5448e-02, -1.5095e-02,  4.1826e-02, -3.8581e-02, -2.3085e-02],[-3.8703e-03,  8.0055e-02,  3.9432e-02, -3.5234e-02, -7.4530e-02],[ 6.6605e-02,  7.1790e-02, -2.6339e-02, -6.3635e-02,  5.8536e-02],[-3.7311e-02,  2.6147e-02, -6.4885e-02, -6.8364e-02, -8.1779e-03]],[[ 5.7778e-04,  6.5419e-02,  5.4751e-02, -2.7986e-02,  2.1922e-02],[-1.1870e-02,  4.0985e-02,  2.4626e-02, -6.6320e-03, -4.1242e-02],[-7.0432e-02,  2.8016e-02, -5.6965e-02, -1.4918e-02, -4.4836e-02],[ 6.8992e-02,  2.6582e-02,  2.8458e-02, -2.8932e-02, -6.5144e-02],[ 3.3061e-02, -9.9823e-03, -6.1666e-02, -5.6478e-02,  1.8484e-02]],[[-3.0624e-02, -4.3539e-02,  6.4283e-02, -3.4875e-02,  7.4699e-02],[-2.9344e-04, -1.3068e-03, -2.6016e-02,  4.0942e-02,  2.7596e-02],[ 3.8677e-02,  1.5856e-03, -6.9073e-02,  6.8789e-02,  6.7985e-02],[ 1.2172e-02,  6.0105e-02, -2.9876e-02,  1.6004e-02,  2.7459e-02],[-6.4051e-02, -4.5696e-02, -2.2469e-02, -7.0772e-02,  3.1664e-02]],[[ 4.2269e-02,  1.6835e-02, -1.6207e-02,  1.9925e-02, -1.3160e-02],[-3.8214e-02, -2.0697e-02,  1.2636e-02,  5.8541e-02,  6.5787e-02],[ 5.4092e-02, -4.1293e-02,  7.1718e-03,  3.5447e-03,  5.9314e-02],[-5.3280e-02,  5.7965e-02, -2.5258e-02,  5.3279e-02,  1.2102e-02],[-6.1438e-02, -4.7414e-02, -6.4839e-02, -4.9603e-02,  3.7255e-02]],[[ 2.8404e-04, -3.0073e-02,  1.2103e-02,  2.6228e-02, -4.6524e-02],[ 6.2227e-02,  7.4788e-02,  7.7427e-02,  6.0245e-02,  4.3996e-02],[-4.6962e-03, -1.5549e-02,  2.2174e-02, -6.1404e-02, -5.2019e-02],[ 1.6444e-02,  3.0072e-02,  4.8501e-02,  7.1959e-02,  7.8801e-02],[-3.8354e-03, -1.2520e-02, -7.9206e-02,  4.9887e-02, -7.5002e-02]]]],requires_grad=True)), ('conv2.bias', Parameter containing:
tensor([-0.0291, -0.0585, -0.0470,  0.0497, -0.0030,  0.0714, -0.0772, -0.0729,-0.0146, -0.0744, -0.0542,  0.0765, -0.0288, -0.0186, -0.0451,  0.0693],requires_grad=True)), ('fc1.weight', Parameter containing:
tensor([[ 0.0127,  0.0351,  0.0101,  ...,  0.0015,  0.0404, -0.0455],[ 0.0154,  0.0499, -0.0289,  ...,  0.0344, -0.0245, -0.0473],[ 0.0072,  0.0226,  0.0097,  ...,  0.0363,  0.0141,  0.0354],...,[ 0.0225,  0.0071, -0.0407,  ..., -0.0101, -0.0304,  0.0344],[ 0.0367,  0.0057, -0.0408,  ..., -0.0393, -0.0016,  0.0063],[ 0.0365, -0.0315,  0.0336,  ..., -0.0208, -0.0220,  0.0388]],requires_grad=True)), ('fc1.bias', Parameter containing:
tensor([ 0.0048, -0.0018,  0.0441, -0.0093, -0.0455, -0.0172,  0.0060,  0.0095,-0.0444, -0.0034, -0.0198, -0.0411,  0.0237, -0.0362, -0.0433,  0.0259,-0.0194,  0.0440, -0.0057,  0.0166,  0.0372, -0.0365,  0.0167,  0.0237,-0.0316, -0.0307, -0.0303, -0.0447,  0.0386, -0.0020, -0.0163, -0.0206,-0.0429,  0.0362, -0.0114,  0.0069, -0.0320,  0.0469,  0.0204,  0.0199,0.0402,  0.0199,  0.0290, -0.0053, -0.0403, -0.0448, -0.0317, -0.0443,-0.0072,  0.0160, -0.0249, -0.0263, -0.0495, -0.0168, -0.0374, -0.0191,0.0002, -0.0181,  0.0465, -0.0054, -0.0368, -0.0173, -0.0434,  0.0306,0.0474,  0.0398,  0.0285, -0.0076, -0.0255,  0.0232,  0.0022,  0.0059,0.0353,  0.0013,  0.0414, -0.0094,  0.0435, -0.0331, -0.0408, -0.0381,0.0160, -0.0089,  0.0269,  0.0171,  0.0205,  0.0466,  0.0360, -0.0251,-0.0087,  0.0258, -0.0364, -0.0393, -0.0077, -0.0196,  0.0485,  0.0066,-0.0440, -0.0421, -0.0475,  0.0283,  0.0257, -0.0366, -0.0414, -0.0026,0.0096,  0.0189,  0.0289, -0.0230, -0.0331, -0.0086,  0.0300,  0.0070,-0.0250,  0.0458, -0.0106, -0.0499, -0.0290,  0.0242, -0.0086, -0.0206],requires_grad=True)), ('fc2.weight', Parameter containing:
tensor([[-0.0868, -0.0417,  0.0663,  ...,  0.0089, -0.0459, -0.0327],[-0.0881,  0.0408, -0.0878,  ...,  0.0712, -0.0304,  0.0808],[ 0.0405,  0.0224, -0.0019,  ..., -0.0402, -0.0458, -0.0392],...,[-0.0641,  0.0234,  0.0442,  ..., -0.0221,  0.0643,  0.0412],[ 0.0731,  0.0759, -0.0588,  ...,  0.0214,  0.0084,  0.0542],[-0.0466,  0.0032,  0.0363,  ..., -0.0843,  0.0305, -0.0020]],requires_grad=True)), ('fc2.bias', Parameter containing:
tensor([ 1.8544e-03,  7.7148e-02, -2.2453e-02, -5.1801e-02,  4.5913e-02,-7.2284e-02, -3.2864e-02, -6.8654e-02, -2.1057e-05,  7.0599e-02,2.5858e-02,  7.3280e-02,  1.1011e-02,  6.3518e-02, -4.0666e-02,-4.1031e-02,  6.6883e-02, -1.2299e-02, -6.6735e-02,  4.0905e-03,-6.5498e-02, -2.6964e-03,  8.2400e-02, -4.4789e-02,  3.2078e-02,-2.4232e-02, -8.0591e-02,  5.7183e-03, -1.8703e-03, -8.2100e-02,-6.2772e-02,  7.0709e-02, -4.4223e-02, -8.6373e-03, -8.9721e-02,-8.7231e-02,  2.1281e-02,  6.5149e-02, -5.8912e-02, -6.0408e-03,3.9662e-02, -5.1060e-02, -5.0107e-02,  7.2825e-02,  8.5803e-02,3.2855e-02, -4.8813e-02, -6.2276e-02,  6.0332e-02, -2.1124e-03,2.0599e-02,  5.5399e-02,  1.8062e-02,  3.8259e-02, -1.0740e-02,-4.9024e-02,  7.9856e-02,  3.3842e-02,  3.2818e-03, -8.6486e-02,6.8716e-04, -3.4156e-02, -6.9502e-02,  6.9801e-02, -1.9957e-02,-8.9092e-02,  8.5563e-02,  6.8908e-02, -1.2464e-03, -5.8290e-02,5.7215e-03, -1.9210e-02,  5.3976e-02, -8.5581e-02, -5.1105e-02,-1.1988e-02, -8.9009e-02,  7.1955e-02, -7.3027e-02, -3.2002e-02,-7.7311e-03,  6.5628e-02,  7.7175e-02, -2.6717e-02],requires_grad=True)), ('fc3.weight', Parameter containing:
tensor([[-5.6418e-02,  8.7256e-03,  1.4857e-03, -4.4241e-02,  2.6271e-02,6.2223e-02,  9.5526e-02,  8.1587e-02, -1.0604e-01, -1.4852e-02,2.0481e-02,  6.4787e-02, -1.7821e-02, -7.5547e-02,  7.8957e-02,-1.3885e-02, -1.0226e-01, -8.2753e-02,  9.8486e-02, -3.4305e-02,2.7100e-03,  1.8032e-02, -1.0715e-01,  4.9688e-03, -9.7491e-02,5.7741e-02,  1.0480e-01, -7.5001e-02, -1.0038e-01, -8.3054e-02,1.4810e-02, -2.5294e-02, -3.5486e-02,  1.0084e-01,  1.2828e-02,-2.8502e-03, -4.3740e-02, -7.8102e-03,  2.1588e-02, -7.5598e-02,-3.3590e-02, -6.8185e-02,  3.3957e-02,  2.1573e-02, -5.8785e-02,6.0361e-02, -8.8535e-02,  6.7021e-02, -9.3063e-02,  1.9414e-02,-6.7784e-02,  3.7093e-02,  7.5324e-03, -1.9678e-02,  5.4143e-02,-8.9511e-02, -9.2640e-02,  6.8033e-02,  2.4691e-02, -4.7642e-02,5.3445e-02, -6.4451e-02,  4.2707e-02, -4.0519e-02, -9.8571e-02,6.6185e-02,  5.7230e-02, -3.7836e-02,  9.9665e-02, -2.2986e-02,-5.7756e-02, -6.8581e-02, -1.1435e-02, -3.2040e-02,  1.0683e-01,-9.7724e-02, -4.2304e-02, -1.2573e-02,  6.7023e-02,  4.1495e-02,7.6838e-02, -6.6680e-02, -7.8055e-02,  5.0132e-02],[ 1.9141e-02,  2.5746e-02,  1.0312e-01,  1.4601e-02,  3.0647e-02,9.3271e-02,  6.4014e-02, -5.4571e-02, -5.6861e-02, -2.6344e-02,9.0764e-02,  5.0952e-02, -9.0798e-02,  2.1471e-02, -1.0136e-01,-8.5665e-02,  2.1246e-02, -2.4271e-02, -6.6701e-02,  7.1168e-02,6.3234e-02, -9.3446e-03,  2.8220e-03, -1.4337e-02, -2.4543e-02,1.0582e-01,  2.0000e-02, -1.0267e-01, -3.5270e-03, -5.9467e-02,-2.3974e-03,  2.7704e-02, -6.6317e-02,  8.0701e-02, -1.0008e-01,-1.0211e-01, -8.1395e-02, -6.5022e-02,  5.9279e-02, -7.2579e-02,-9.5273e-02, -6.4547e-02, -8.0597e-02,  8.4433e-03, -4.7848e-02,9.7750e-02,  9.3517e-03,  7.4619e-02, -3.9448e-02, -8.5778e-02,-9.5482e-02, -5.5330e-03, -1.0796e-01,  6.7140e-02, -2.9456e-02,-1.8318e-02, -4.2094e-02,  1.8795e-02,  1.0150e-02,  4.3263e-02,-7.2860e-02,  7.8749e-02, -5.2190e-03,  1.5920e-02,  5.5891e-02,-4.6358e-02,  9.4703e-02,  8.4561e-02,  5.2485e-02,  7.6034e-02,-2.0514e-03,  5.1548e-02, -2.3387e-03, -1.0369e-01, -1.7051e-03,9.5608e-02, -7.4236e-02,  8.8553e-02, -7.3277e-02, -6.1156e-02,-4.4343e-02,  1.5902e-02,  8.3211e-02, -4.9491e-02],[-5.4914e-03, -8.3233e-02,  2.0474e-02, -7.9890e-02, -2.6837e-02,9.9609e-02, -6.1847e-02,  8.2810e-03, -7.8208e-02, -2.2464e-02,-8.8908e-02,  8.8578e-02,  7.5675e-02, -1.2950e-02,  9.0193e-02,-2.1504e-02,  8.8890e-02,  9.8521e-02, -7.5039e-02,  1.0464e-01,9.4199e-02, -3.8682e-02,  8.7018e-02,  8.5465e-02, -7.7055e-02,-7.1284e-02, -1.0897e-01, -5.4695e-03, -7.9368e-02,  3.3380e-02,7.2154e-02,  9.5790e-02,  6.1063e-04, -1.7953e-02,  6.1940e-02,-7.8037e-02,  9.0474e-02,  9.4975e-02,  4.7951e-02,  2.9325e-02,2.9166e-04,  5.0427e-02,  4.9875e-02,  1.9344e-02, -1.5598e-02,-8.8184e-02,  4.9534e-02,  4.1846e-02, -3.4013e-02, -5.9683e-02,-7.4203e-02,  4.6901e-02, -5.7360e-02,  3.9085e-02, -1.0259e-01,5.0026e-02,  9.3175e-02, -4.9197e-02, -9.1434e-02, -6.5976e-02,9.9470e-02,  2.5012e-02,  2.3345e-02, -7.4206e-02,  9.9779e-02,-9.9292e-02,  9.1207e-02, -1.8750e-02, -8.7222e-02,  4.5009e-02,6.2097e-02,  1.1889e-02, -3.1094e-02,  6.2701e-02, -3.6896e-02,7.9025e-03,  3.9122e-02, -6.6788e-02,  5.8422e-02, -1.8931e-02,4.7233e-02,  7.9713e-03,  6.5758e-02,  2.4974e-02],[ 9.3710e-02,  6.3587e-03, -1.1848e-02,  4.5289e-02, -2.2177e-02,-7.1277e-02,  4.7252e-02,  1.6109e-02, -6.7931e-02, -1.7756e-02,2.0796e-02,  7.4046e-02, -1.0572e-01, -9.9937e-02,  6.4527e-04,7.9186e-02, -2.9363e-02,  9.3051e-02,  2.8610e-02,  5.4603e-02,1.7861e-02,  5.2665e-02,  9.7904e-02, -2.5723e-02, -3.2497e-02,9.0306e-02, -8.9747e-02,  7.0985e-02,  4.8148e-02,  8.0181e-02,-7.6122e-02, -1.0335e-01, -8.7064e-02,  9.8974e-02, -5.0385e-02,4.0987e-02,  1.0069e-01,  4.4886e-02, -6.0288e-02,  2.8446e-02,9.9679e-02,  5.8738e-02, -8.6646e-02,  4.9163e-02, -1.0731e-01,-8.7259e-02,  8.3456e-02,  6.8349e-02, -8.7784e-02, -8.0034e-02,4.8366e-02,  4.0478e-02,  5.7153e-03, -5.4092e-02,  5.2010e-03,8.2633e-02, -1.0290e-01,  5.6955e-02, -9.8502e-02,  4.5809e-03,4.4042e-02,  9.9767e-03,  5.3250e-02, -2.6768e-02, -7.9477e-02,-1.0056e-01, -1.5802e-03, -6.9598e-03, -6.3054e-02,  1.0204e-01,-9.7165e-02, -9.1548e-02,  2.5512e-02, -5.9958e-02, -2.2757e-02,-7.9634e-03, -1.0523e-01, -6.3195e-02, -1.2414e-02,  1.1663e-02,-8.9895e-02, -8.0842e-02,  1.0750e-01, -3.0337e-02],[ 5.6841e-02,  8.6600e-02,  1.7552e-02, -5.7772e-03, -4.4277e-02,-4.1141e-03, -4.0124e-02,  3.6352e-02, -8.9301e-02,  4.6127e-02,7.3090e-02, -9.3387e-02, -3.1823e-02,  4.5977e-02,  1.0641e-01,-2.9397e-02,  4.1653e-02,  5.7302e-03,  4.0278e-02, -4.2121e-02,-1.0518e-01,  9.9239e-02, -2.4550e-02,  5.3995e-02, -4.9111e-02,-1.2914e-02,  5.2915e-02,  8.5013e-02,  6.0425e-02,  9.0397e-02,-7.9633e-02,  3.0904e-02,  8.4106e-02,  7.7185e-02, -1.1237e-02,-3.8323e-02,  4.4578e-03, -4.1946e-02, -9.6759e-02, -1.2592e-02,-4.1569e-02, -1.0844e-01, -5.4256e-02,  2.7450e-02, -6.3376e-02,-1.0629e-01, -2.7586e-02, -4.1033e-02, -1.0556e-01, -1.6890e-03,-1.0441e-01, -2.9892e-02,  9.5217e-03,  8.4235e-02, -1.0753e-01,2.1481e-02,  5.6448e-03, -9.0530e-02,  1.0382e-01, -2.1958e-02,4.4257e-02,  4.6737e-02, -6.9540e-02, -5.2136e-02,  6.0439e-02,4.6486e-02, -7.2450e-02, -1.1493e-02,  1.7681e-02,  6.1643e-02,7.3240e-02,  2.1737e-02,  1.2148e-02, -9.1230e-02,  7.5856e-02,5.2213e-02, -2.0461e-02, -5.2718e-02, -1.0508e-01,  1.0872e-02,-1.5246e-03, -2.6219e-02,  8.0384e-03,  1.4495e-02],[-5.3898e-02, -6.2903e-02,  9.3278e-02,  5.2151e-02,  2.4894e-02,7.4073e-02,  5.3318e-02, -2.8024e-02, -1.0801e-01,  1.0287e-01,4.7442e-02,  5.0882e-02, -2.4797e-02, -9.7050e-02,  8.4431e-02,6.6385e-02,  1.0674e-01, -1.0502e-01, -5.5089e-03, -2.1849e-02,4.1641e-02,  6.3477e-02, -7.0544e-02, -1.0295e-01, -6.6842e-02,8.4617e-03,  6.7929e-02,  5.4032e-02, -1.3090e-02, -2.8971e-02,-4.3182e-02, -4.4062e-02, -9.7478e-02, -6.2066e-02, -5.4336e-02,5.0292e-02,  7.3119e-02,  8.2924e-02,  9.4342e-02,  4.5792e-02,-4.1454e-02,  6.4998e-02, -5.4642e-02,  1.6673e-02, -3.3706e-02,-9.6359e-02,  1.7345e-04, -9.2701e-02,  8.3176e-03,  2.0206e-02,-3.3897e-03, -4.6855e-02,  5.2317e-02, -4.5804e-02,  2.2652e-02,-1.0233e-01, -3.7416e-03, -7.8552e-02, -1.7273e-02, -2.1090e-02,-7.0454e-03,  6.4824e-02,  1.0302e-01, -5.9749e-03,  1.0142e-01,7.1721e-02, -3.2308e-02,  1.0354e-01, -2.1937e-02,  9.5663e-02,-5.3376e-03, -8.3901e-02,  2.2311e-02,  4.4284e-02, -1.1621e-03,-1.9146e-02,  9.5266e-02,  4.9740e-02,  8.7377e-02, -6.9876e-02,7.4907e-02, -1.7911e-02,  7.3294e-02,  7.7561e-02],[-1.0208e-01,  8.0829e-02, -7.1439e-02,  2.2382e-02,  7.3827e-02,-5.1018e-02, -6.3343e-02, -2.9540e-02,  9.3982e-02,  4.3037e-02,8.5199e-02, -5.6971e-02,  2.5385e-02,  7.3944e-02,  3.9802e-02,7.6630e-02, -1.4518e-02,  5.0097e-02,  4.9435e-02,  6.8677e-02,-1.9400e-02,  9.0476e-03,  9.9804e-02, -9.8907e-02,  7.7342e-02,5.8050e-02, -1.0695e-02,  2.4084e-02,  6.4561e-02, -1.5142e-02,-2.9764e-02,  1.3710e-03,  1.3472e-02, -4.0897e-03,  3.9828e-02,1.0682e-01, -3.4013e-02, -7.1825e-02,  1.0256e-01, -4.8058e-02,-7.9164e-02, -3.1616e-02,  1.4847e-02, -5.4025e-02,  1.0359e-01,-7.2879e-02, -5.5342e-02,  9.1632e-02,  9.1646e-02, -4.5751e-02,6.7954e-02,  3.2295e-02,  6.9718e-03,  9.7817e-02,  1.0138e-01,5.1222e-02, -6.8627e-02, -4.2274e-02, -5.4111e-02,  2.0167e-02,9.3684e-02, -7.3912e-02, -4.8360e-02,  2.9101e-02,  5.3984e-02,-6.4280e-02, -7.3429e-02,  6.6071e-02, -4.8896e-02, -2.6180e-02,8.5460e-02,  7.1012e-02,  2.1531e-02, -7.9570e-02,  9.2200e-02,-9.2980e-02,  7.6317e-02, -4.6281e-02, -1.0742e-01,  3.3758e-02,-1.0100e-01, -5.6719e-02, -1.4088e-02,  6.6273e-02],[ 5.3003e-02, -8.5680e-02, -3.2159e-04, -6.6686e-02,  1.0357e-01,7.7528e-02, -7.7695e-02, -3.7343e-02, -6.4867e-02, -3.4528e-02,3.8912e-02, -2.9453e-02,  7.2781e-02,  8.9834e-02, -7.9001e-02,-8.3703e-02,  5.7901e-02,  6.9107e-02,  7.9265e-02,  4.9187e-02,5.0513e-02, -5.4097e-02, -3.2582e-02,  6.4819e-02,  1.9896e-02,-7.3084e-02, -8.9256e-02,  4.9840e-02, -5.9509e-02,  5.3884e-02,-3.8777e-02,  6.0162e-02,  1.0393e-01, -2.5981e-02,  3.4439e-02,7.1359e-02, -8.5326e-02, -8.6582e-02,  2.3416e-02, -4.4381e-02,-3.0652e-03,  3.3561e-02,  4.0531e-02,  6.6637e-02, -1.0751e-01,-2.0587e-02, -5.7200e-02,  8.0107e-02, -9.5006e-02,  8.8297e-02,5.6524e-02,  5.3064e-02, -5.4679e-02, -6.5577e-02,  2.3586e-02,2.7163e-02,  3.3865e-03,  9.4055e-02, -4.6757e-02, -1.8562e-02,-7.2532e-02,  4.8080e-02, -9.0010e-02, -1.0510e-01, -7.9523e-02,-7.7586e-02, -8.7399e-02, -2.3291e-02,  5.0533e-02,  9.0525e-02,5.7866e-02,  9.3118e-02,  6.4315e-02, -9.9633e-02,  3.1986e-02,8.2956e-02,  7.0101e-02, -2.0538e-02,  8.6799e-02, -6.9437e-02,1.0641e-01,  1.0426e-01,  8.0581e-02, -8.0374e-02],[ 3.2695e-02, -3.4856e-02,  6.2409e-02, -1.3378e-02,  6.0558e-03,1.4727e-02, -8.5913e-02,  1.0691e-01,  7.7849e-02, -4.7844e-02,6.0482e-02, -7.5180e-03, -4.0028e-02,  6.0692e-02, -5.5052e-02,3.3449e-02, -8.4209e-02, -1.3423e-02,  9.3352e-02,  1.3072e-02,-5.9021e-04,  7.6759e-02,  8.5805e-02, -2.4546e-02, -9.6939e-02,-4.4700e-02,  5.6368e-02, -1.1727e-02,  5.2196e-02,  6.8510e-02,4.8723e-02, -3.9295e-02, -3.2981e-02, -3.3015e-02, -1.3612e-02,-7.1090e-02, -7.5796e-02, -4.9240e-02,  1.4497e-02,  2.2467e-02,-4.2246e-03,  7.9742e-02,  9.8798e-02, -1.0770e-01, -1.0561e-01,8.5984e-02, -9.2008e-02,  5.3294e-02, -5.5027e-02, -4.8945e-02,1.9929e-02,  8.5859e-02,  6.2954e-02,  2.9138e-03,  4.9239e-02,2.2325e-02, -9.6379e-02,  1.4460e-02, -3.5029e-02, -4.5814e-02,-7.4607e-02,  7.0636e-02,  6.4684e-02, -2.8305e-02,  1.7848e-03,-1.5799e-02,  6.4801e-02,  5.6922e-02, -9.3420e-02, -2.4182e-02,4.9939e-02, -4.6919e-02, -9.4452e-02, -6.2229e-02,  2.6796e-02,-1.9359e-02, -1.0534e-01, -1.0308e-01,  6.7256e-02, -1.0804e-01,-5.2163e-02, -3.9063e-02, -1.1436e-02, -4.9283e-02],[ 3.7929e-02, -9.2263e-02,  8.3047e-02, -1.4673e-03,  1.0762e-01,4.5225e-02, -6.9877e-02, -8.5745e-02,  4.3992e-02,  8.6473e-02,-3.9794e-02, -6.4171e-02,  2.1542e-02,  1.5011e-03, -8.4764e-02,7.6233e-02,  1.4007e-02, -4.1551e-02,  6.5961e-02, -9.7566e-02,-9.3359e-02,  3.9779e-02,  3.7884e-03,  2.0744e-02,  1.0474e-01,4.3060e-02, -2.9415e-02, -1.3947e-02, -1.0270e-01, -1.4549e-02,-3.3965e-02,  7.2888e-02, -1.5545e-02, -2.1496e-02, -9.5266e-02,4.0747e-02, -2.9195e-02, -6.9564e-02,  8.0910e-02, -4.2163e-02,-1.3920e-02, -3.4202e-02,  8.5124e-02,  8.0781e-02, -4.7417e-02,1.0826e-01,  2.4278e-02,  3.6846e-02, -9.6196e-02,  1.2560e-02,9.7764e-02,  1.0009e-01,  1.0832e-01,  2.5618e-03,  7.1625e-02,-7.1904e-02, -6.2205e-02,  5.8912e-02,  2.3969e-02, -9.6306e-02,2.9396e-02,  3.1466e-02,  6.3138e-02,  3.7381e-02,  1.0424e-01,-1.0732e-01, -1.0593e-01,  3.2480e-02, -1.0435e-01, -1.3005e-02,5.2493e-02, -6.7261e-02, -7.5851e-02, -4.7852e-05,  1.2423e-02,-7.6787e-02, -1.0262e-02, -1.6454e-02,  9.6626e-02, -1.0664e-01,-8.2998e-02, -2.1841e-02, -5.2301e-02, -1.5127e-02]],requires_grad=True)), ('fc3.bias', Parameter containing:
tensor([ 0.0554,  0.0078,  0.0239,  0.0951,  0.0102,  0.0927,  0.0383,  0.0847,0.0910, -0.0069], requires_grad=True))]

测试随机输入32×32。注：这个网络（LeNet）期望的输入大小是32×32，如果使用MNIST数据集来训练这个网络，请把图片大小重新调整到32×32。

input = torch.randn(1, 1, 32, 32)
out = net(input)
print(out)

tensor([[-0.0384,  0.0331,  0.0275,  0.0812, -0.0870,  0.1044,  0.0813,  0.0467,0.0679,  0.0307]], grad_fn=<AddmmBackward>)

将所有参数的梯度缓存清零，然后进行随机梯度的的反向传播：

net.zero_grad()
out.backward(torch.randn(1, 10))

Note
torch.nn 只支持小批量输入。整个 torch.nn 包都只支持小批量样本，而不支持单个样本。例如，nn.Conv2d 接受一个4维的张量， 每一维分别是sSamples * nChannels * Height * Width（样本数*通道数*高*宽）。如果你有单个样本，只需使用 input.unsqueeze(0) 来添加其它的维数。

损失函数

一个损失函数接受一对 (output, target) 作为输入，计算一个值来估计网络的输出和目标值相差多少。
nn包中有很多不同的损失函数。 nn.MSELoss是一个比较简单的损失函数，它计算输出和目标间的均方误差，例如：

output = net(input)
target = torch.randn(10)  # 随机值作为样例
target = target.view(1, -1)  # 使target和output的shape相同
criterion = nn.MSELoss()loss = criterion(output, target)
print(loss)

tensor(0.4847, grad_fn=<MseLossBackward>)

反向传播

调用loss.backward()获得反向传播的误差。

但是在调用前需要清除已存在的梯度，否则梯度将被累加到已存在的梯度。

现在，我们将调用loss.backward()，并查看conv1层的偏差（bias）项在反向传播前后的梯度。

net.zero_grad()     # 清除梯度print('conv1.bias.grad before backward')
print(net.conv1.bias.grad)loss.backward()print('conv1.bias.grad after backward')
print(net.conv1.bias.grad)

conv1.bias.grad before backward
tensor([0., 0., 0., 0., 0., 0.])
conv1.bias.grad after backward
tensor([ 0.0051,  0.0042,  0.0026,  0.0152, -0.0040, -0.0036])

更新权重

在实践中最简单的权重更新规则是随机梯度下降（SGD）：

weight = weight - learning_rate * gradient
我们可以使用简单的Python代码实现这个规则：

learning_rate = 0.01
for f in net.parameters():f.data.sub_(f.grad.data * learning_rate)

但是当使用神经网络是想要使用各种不同的更新规则时，比如SGD、Nesterov-SGD、Adam、RMSPROP等，PyTorch中构建了一个包torch.optim实现了所有的这些规则。使用它们非常简单：

import torch.optim as optim# create your optimizer
optimizer = optim.SGD(net.parameters(), lr=0.01)# in your training loop:
optimizer.zero_grad()   # zero the gradient buffers
output = net(input)
loss = criterion(output, target)
loss.backward()
optimizer.step()    # Does the update

Pytorch 构建简单Neural Networks相关推荐

keras构建卷积神经网络（CNN(Convolutional Neural Networks)）进行图像分类模型构建和学习
keras构建卷积神经网络(CNN(Convolutional Neural Networks))进行图像分类模型构建和学习全连接神经网络(Fully connected neural networ ...
《Intriguing properties of neural networks》代码实现——Pytorch
<Intriguing properties of neural networks>代码实现--Pytorch 一.代码实现 #导入库 import torch import torch. ...
Non-local Neural Networks：非局部神经网络
论文地址:https://openaccess.thecvf.com/content_cvpr_2018/papers/Wang_Non-Local_Neural_Networks_CVPR_2018 ...
SPINN: Synergistic Progressive Inferenceof Neural Networks over Device and Cloud
题目:SPINN: Synergistic Progressive Inferenceof Neural Networks over Device and Cloud SPINN:设备和云上神经网络的 ...
[论文翻译]-A Comprehensive Survey on Graph Neural Networks《图神经网络GNN综述》
文章目录摘要 1 简介 1.1 GNN简史 1.2 Related surveys on graph neural networks 1.3 Graph neural networks vs. ne ...
【阅读】A Comprehensive Survey on Electronic Design Automation and Graph Neural Networks——EDA+GNN综述翻译
转载请注明出处:小锋学长生活大爆炸[xfxuezhang.blog.csdn.net] Paper原文:https://dl.acm.org/doi/pdf/10.1145/3543853 ...
论文翻译：A Comprehensive Survey on Graph Neural Networks
论文翻译:图神经网络综合研究 arXiv:1901.00596v1 文章目录论文翻译:图神经网络综合研究 1 引言 2 定义 3 分类和框架 3.1 GNN的分类 3.2 框架 4 图卷积网络 4. ...
【阅读】A Comprehensive Survey on Distributed Training of Graph Neural Networks——翻译
转载请注明出处:小锋学长生活大爆炸[xfxuezhang.cn] (本文中,涉及到公式部分的翻译不准确,请看对应原文.) 另一篇:[阅读]Distributed Graph Neural Networ ...
[HyperGraph专题]HGNN+:General Hypergraph Neural Networks
最近在做和超图相关的工作,自己主要偏编码的工作,但在敲代码和参考其他现有超图库的过程中发现了一个叫DeepHypergrah的库,里面涉及了很多和深度学习相关的内容,用到了pytorch.scipy. ...

Pytorch 构建简单Neural Networks