NNDL 作业9：分别使用numpy和pytorch实现BPTT

6-1：推导RNN反向传播算法BPTT.

6-2P：设计简单RNN模型，分别用Numpy、Pytorch实现反向传播算子，并代入数值测试.

6-1：推导RNN反向传播算法BPTT.

在RNN中，输入序列为 $\left \{ x^{(1)},x^{(2)},...,x^{(T)} \right \}$ \left \{ x^{(1)},x^{(2)},...,x^{(T)} \right \}，输出序列为 $\left \{ \hat{y}^{(1)},\hat{y}^{(2)},...,\hat{y}^{(T)} \right \}$ ，标签序列为 $\left \{ y^{(1)},y^{(2)},...,y^{(T)} \right \}$ \left \{ y^{(1)},y^{(2)},...,y^{(T)} \right \} ，其中的 $x^{(i)},y^{(i)},\hat{y}^{(i)}$ 均为列向量，模型的更新方式为：

$a^{(t)}=b+Wh^{(t-1)}+Ux^{(t)}$

$h^{(t)}=tanh(a^{(t)})$

$o^{(t)}=c+Vh^{(t)}$

$\hat{y}^{(t)}=softmax(o^{(t)})$

损失函数为负对数似然函数，总体损失为每个时间步的损失之和：

$L( \left \{ x^{(1)},x^{(2)},...,x^{(T)} \right \},\left \{ y^{(1)},y^{(2)},...,y^{(T)} \right \})$

$= \sum_{t}^{}L^{(t)}$

$=-\sum_{t}^{}log _{Pmodel}(y^{(t)}|\left \{ x^{(1)},x^{(2)},...,x^{(T)} \right \})$

需要更新的参数有 $b,W,U,V,c$ ，根据链式求导法则，先对 $o^{(t)}$ 进行求导。假设 $o^{(t)}$ 是 n 维向量，则 $y^{(i)},\hat{y}^{(i)}$ 也是 n维列向量， t 时刻的标签 $y^{(t)}$ 对应其中的第 i 维，则对应地定义 $1_{i,y(t)}$ 这样的向量：第 i 维为1，其余为0，则 :

$\bigtriangledown _{o^{(t)}}L=\frac{\partial L}{\partial o^{(t)}}$

$=-\frac{\partial L}{\partial L^{(t)}}\frac{\partial L^{(t)}}{\partial o^{(t)}}$

$=\hat{y}^{(t)}-1_{i,y(t)}$

其结果为n维列向量，考虑到序列之间的关系，根据公式 :

$\frac{\partial Ax}{\partial x}=A^{T}$

先计算最后的隐状态 $h^{(T)}$ 的梯度：

$\bigtriangledown _{h^{(T)}}L=-\frac{\partial o^{(T)}}{\partial h^{(T)}}\frac{\partial L}{\partial o^{(T)}}=V^{T}\bigtriangledown _{o^{(t)}}L$

这里对向量对向量求导结果，统一使用分母布局形式记录， n 维列向量 y对 m 维列向量 x 的导数矩阵维度为m×n ，即：

$\frac{dy}{dx}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} & \frac{\partial y_{2}}{\partial x_{1}} & \cdots & \frac{\partial y_{n}}{\partial x_{1}}\\ \frac{\partial y_{1}}{\partial x_{2}} & \frac{\partial y_{2}}{\partial x_{2}} & \cdots & \frac{\partial y_{n}}{\partial x_{2}}\\ \cdots & \cdots & \cdots & \cdots \\ \frac{\partial y_{1}}{\partial x_{m}} & \frac{\partial y_{2}}{\partial x_{m}} & \cdots & \frac{\partial y_{n}}{\partial x_{m}} \end{bmatrix}$

这里经常用到这样一个式子，对于 $y=f(x) ,z=g(y)$ ，其中z为标量，则

$\frac{dz}{dx}=\frac{dy}{dx}\frac{dz}{dy}$

这样写是由矩阵元素摆放位置决定的。对于 τ 之前的隐状态 $h^{(t)}$ ，根据迭代系， $h^{(t+1)}$ 和 $o^{(t)}$ 与之相关，故：

$\bigtriangledown _{h^{(t)}}L=-\frac{\partial h^{(t+1)}}{\partial h^{(t)}}\frac{\partial L}{\partial h^{(t+1)}}+\frac{\partial o^{(t)}}{\partial h^{(t)}}\frac{\partial L}{\partial o^{(t)}}$

$=\frac{\partial h^{(t+1)}}{\partial h^{(t)}}\bigtriangledown _{h^{(t+1)}}L+V^{T}\frac{\partial L}{\partial o^{(t)}}$

其中：

$\frac{\partial h^{(t+1)}}{\partial h^{(t)}}=W^{T}diag(1-(h^{(t+1)^{}})^{2})$

那么 :

$\bigtriangledown _{h^{(t)}}L=W^{T}diag(1-(h^{(t+1)^{}})^{2})\bigtriangledown _{h^{(t+1)}}L+V^{T}\bigtriangledown _{o^{(t)}}L$

也就是说， $h^{(t)}$ 的梯度可以递归的进行求解。接下来，我们写出b,W,U,V,c的梯度即可：

$\bigtriangledown _{b}L=\sum_{t}^{}\frac{\partial h^{(t)}}{\partial b}\frac{\partial L}{\partial h^{(t)}}$

$=\sum_{t}^{}diag(1-(h^{(t)^{}})^{2})\bigtriangledown _{h^{(t)}}L$

$\bigtriangledown _{c}L=\sum_{t}^{}\frac{\partial o^{(t)}}{\partial c}\frac{\partial L}{\partial o^{(t)}}$

$=\sum_{t}^{}\bigtriangledown _{o^{(t)}}L$

在计算时，需要将中间变量进行拆解，最后进行合成即可。

$\bigtriangledown _{V}L=\sum_{t}^{}\sum_{t}^{}\frac{\partial L}{\partial o_{i}^{(t)}}\frac{\partial o_{i}^{(t)}}{\partial V}$

$=\sum_{t}^{}\sum_{t}^{}\frac{\partial L}{\partial o_{i}^{(t)}}\begin{bmatrix} \cdots \\ \cdots\\ h^{(t)^{T}}\\ \cdots \end{bmatrix}$

$=\sum_{t}^{}(\bigtriangledown _{o^{(t)}}L)h^{(t)^{T}}$

类似地

$\bigtriangledown _{W}L=\sum_{t}^{}\sum_{t}^{}\frac{\partial L}{\partial h_{i}^{(t)}}\frac{\partial h_{i}^{(t)}}{\partial W}$

$=\sum_{t}^{}diag(1-(h^{(t)^{}})^{2})(\bigtriangledown _{h^{(t)}}L)h^{(t-1)^{T}}$

$\bigtriangledown _{U}L=\sum_{t}^{}\sum_{t}^{}\frac{\partial L}{\partial h_{i}^{(t)}}\frac{\partial h_{i}^{(t)}}{\partial U}$

$=\sum_{t}^{}diag(1-(h^{(t)^{}})^{2})(\bigtriangledown _{h^{(t)}}L)x^{(t)^{T}}$

6-2P：设计简单RNN模型，分别用Numpy、Pytorch实现反向传播算子，并代入数值测试.

import torch
import numpy as npclass RNNCell:def __init__(self, weight_ih, weight_hh,bias_ih, bias_hh):self.weight_ih = weight_ihself.weight_hh = weight_hhself.bias_ih = bias_ihself.bias_hh = bias_hhself.x_stack = []self.dx_list = []self.dw_ih_stack = []self.dw_hh_stack = []self.db_ih_stack = []self.db_hh_stack = []self.prev_hidden_stack = []self.next_hidden_stack = []# temporary cacheself.prev_dh = Nonedef __call__(self, x, prev_hidden):self.x_stack.append(x)next_h = np.tanh(np.dot(x, self.weight_ih.T)+ np.dot(prev_hidden, self.weight_hh.T)+ self.bias_ih + self.bias_hh)self.prev_hidden_stack.append(prev_hidden)self.next_hidden_stack.append(next_h)# clean cacheself.prev_dh = np.zeros(next_h.shape)return next_hdef backward(self, dh):x = self.x_stack.pop()prev_hidden = self.prev_hidden_stack.pop()next_hidden = self.next_hidden_stack.pop()d_tanh = (dh + self.prev_dh) * (1 - next_hidden ** 2)self.prev_dh = np.dot(d_tanh, self.weight_hh)dx = np.dot(d_tanh, self.weight_ih)self.dx_list.insert(0, dx)dw_ih = np.dot(d_tanh.T, x)self.dw_ih_stack.append(dw_ih)dw_hh = np.dot(d_tanh.T, prev_hidden)self.dw_hh_stack.append(dw_hh)self.db_ih_stack.append(d_tanh)self.db_hh_stack.append(d_tanh)return self.dx_listif __name__ == '__main__':np.random.seed(123)torch.random.manual_seed(123)np.set_printoptions(precision=6, suppress=True)rnn_PyTorch = torch.nn.RNN(4, 5).double()rnn_numpy = RNNCell(rnn_PyTorch.all_weights[0][0].data.numpy(),rnn_PyTorch.all_weights[0][1].data.numpy(),rnn_PyTorch.all_weights[0][2].data.numpy(),rnn_PyTorch.all_weights[0][3].data.numpy())nums = 3x3_numpy = np.random.random((nums, 3, 4))x3_tensor = torch.tensor(x3_numpy, requires_grad=True)h3_numpy = np.random.random((1, 3, 5))h3_tensor = torch.tensor(h3_numpy, requires_grad=True)dh_numpy = np.random.random((nums, 3, 5))dh_tensor = torch.tensor(dh_numpy, requires_grad=True)h3_tensor = rnn_PyTorch(x3_tensor, h3_tensor)h_numpy_list = []h_numpy = h3_numpy[0]for i in range(nums):h_numpy = rnn_numpy(x3_numpy[i], h_numpy)h_numpy_list.append(h_numpy)h3_tensor[0].backward(dh_tensor)for i in reversed(range(nums)):rnn_numpy.backward(dh_numpy[i])print("numpy_hidden :\n", np.array(h_numpy_list))print("torch_hidden :\n", h3_tensor[0].data.numpy())print("-----------------------------------------------")print("dx_numpy :\n", np.array(rnn_numpy.dx_list))print("dx_torch :\n", x3_tensor.grad.data.numpy())print("------------------------------------------------")print("dw_ih_numpy :\n",np.sum(rnn_numpy.dw_ih_stack, axis=0))print("dw_ih_torch :\n",rnn_PyTorch.all_weights[0][0].grad.data.numpy())print("------------------------------------------------")print("dw_hh_numpy :\n",np.sum(rnn_numpy.dw_hh_stack, axis=0))print("dw_hh_torch :\n",rnn_PyTorch.all_weights[0][1].grad.data.numpy())print("------------------------------------------------")print("db_ih_numpy :\n",np.sum(rnn_numpy.db_ih_stack, axis=(0, 1)))print("db_ih_torch :\n",rnn_PyTorch.all_weights[0][2].grad.data.numpy())print("-----------------------------------------------")print("db_hh_numpy :\n",np.sum(rnn_numpy.db_hh_stack, axis=(0, 1)))print("db_hh_torch :\n",rnn_PyTorch.all_weights[0][3].grad.data.numpy())

numpy_hidden :[[[ 0.4686   -0.298203  0.741399 -0.446474  0.019391][ 0.365172 -0.361254  0.426838 -0.448951  0.331553][ 0.589187 -0.188248  0.684941 -0.45859   0.190099]][[ 0.146213 -0.306517  0.297109  0.370957 -0.040084][-0.009201 -0.365735  0.333659  0.486789  0.061897][ 0.030064 -0.282985  0.42643   0.025871  0.026388]][[ 0.225432 -0.015057  0.116555  0.080901  0.260097][ 0.368327  0.258664  0.357446  0.177961  0.55928 ][ 0.103317 -0.029123  0.182535  0.216085  0.264766]]]
torch_hidden :[[[ 0.4686   -0.298203  0.741399 -0.446474  0.019391][ 0.365172 -0.361254  0.426838 -0.448951  0.331553][ 0.589187 -0.188248  0.684941 -0.45859   0.190099]][[ 0.146213 -0.306517  0.297109  0.370957 -0.040084][-0.009201 -0.365735  0.333659  0.486789  0.061897][ 0.030064 -0.282985  0.42643   0.025871  0.026388]][[ 0.225432 -0.015057  0.116555  0.080901  0.260097][ 0.368327  0.258664  0.357446  0.177961  0.55928 ][ 0.103317 -0.029123  0.182535  0.216085  0.264766]]]
-----------------------------------------------
dx_numpy :[[[-0.643965  0.215931 -0.476378  0.072387][-1.221727  0.221325 -0.757251  0.092991][-0.59872  -0.065826 -0.390795  0.037424]][[-0.537631 -0.303022 -0.364839  0.214627][-0.815198  0.392338 -0.564135  0.217464][-0.931365 -0.254144 -0.561227  0.164795]][[-1.055966  0.249554 -0.623127  0.009784][-0.45858   0.108994 -0.240168  0.117779][-0.957469  0.315386 -0.616814  0.205634]]]
dx_torch :[[[-0.643965  0.215931 -0.476378  0.072387][-1.221727  0.221325 -0.757251  0.092991][-0.59872  -0.065826 -0.390795  0.037424]][[-0.537631 -0.303022 -0.364839  0.214627][-0.815198  0.392338 -0.564135  0.217464][-0.931365 -0.254144 -0.561227  0.164795]][[-1.055966  0.249554 -0.623127  0.009784][-0.45858   0.108994 -0.240168  0.117779][-0.957469  0.315386 -0.616814  0.205634]]]
------------------------------------------------
dw_ih_numpy :[[3.918335 2.958509 3.725173 4.157478][1.261197 0.812825 1.10621  0.97753 ][2.216469 1.718251 2.366936 2.324907][3.85458  3.052212 3.643157 3.845696][1.806807 1.50062  1.615917 1.521762]]
dw_ih_torch :[[3.918335 2.958509 3.725173 4.157478][1.261197 0.812825 1.10621  0.97753 ][2.216469 1.718251 2.366936 2.324907][3.85458  3.052212 3.643157 3.845696][1.806807 1.50062  1.615917 1.521762]]
------------------------------------------------
dw_hh_numpy :[[ 2.450078  0.243735  4.269672  0.577224  1.46911 ][ 0.421015  0.372353  0.994656  0.962406  0.518992][ 1.079054  0.042843  2.12169   0.863083  0.757618][ 2.225794  0.188735  3.682347  0.934932  0.955984][ 0.660546 -0.321076  1.554888  0.833449  0.605201]]
dw_hh_torch :[[ 2.450078  0.243735  4.269672  0.577224  1.46911 ][ 0.421015  0.372353  0.994656  0.962406  0.518992][ 1.079054  0.042843  2.12169   0.863083  0.757618][ 2.225794  0.188735  3.682347  0.934932  0.955984][ 0.660546 -0.321076  1.554888  0.833449  0.605201]]
------------------------------------------------
db_ih_numpy :[7.568411 2.175445 4.335336 6.820628 3.51003 ]
db_ih_torch :[7.568411 2.175445 4.335336 6.820628 3.51003 ]
-----------------------------------------------
db_hh_numpy :[7.568411 2.175445 4.335336 6.820628 3.51003 ]
db_hh_torch :[7.568411 2.175445 4.335336 6.820628 3.51003 ]

ref：RNN的反向传播-BPTT - 知乎 (zhihu.com)

L5W1作业1 手把手实现循环神经网络_追寻远方的人的博客-CSDN博客