编程环境

Python：3.9
Pytorch：1.11.0

Bug描述

Traceback (most recent call last):File "E:\Anaconda\envs\torch_c_13\lib\site-packages\IPython\core\interactiveshell.py", line 3343, in run_codeexec(code_obj, self.user_global_ns, self.user_ns)File "<ipython-input-2-18bbaf295f9c>", line 1, in <module>runfile('E:/Code/AEs by PyTorch/AEsingle_train_test_temp.py', wdir='E:/Code/AEs by PyTorch')File "E:\SoftWare\PyCharm\PyCharm 2021.2.3\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 198, in runfilepydev_imports.execfile(filename, global_vars, local_vars)  # execute the scriptFile "E:\SoftWare\PyCharm\PyCharm 2021.2.3\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfileexec(compile(contents+"\n", file, 'exec'), glob, loc)File "E:/Code/AEs by PyTorch/AEsingle_train_test_temp.py", line 205, in <module>train_ae_x_h2 = AEtrain(AEmodel2, train_ae_x_h1, 10, "AEmodel2")File "E:/Code/AEs by PyTorch/AEsingle_train_test_temp.py", line 95, in AEtrainloss.backward()File "E:\Anaconda\envs\torch_c_13\lib\site-packages\torch\tensor.py", line 195, in backwardtorch.autograd.backward(self, gradient, retain_graph, create_graph)File "E:\Anaconda\envs\torch_c_13\lib\site-packages\torch\autograd\__init__.py", line 99, in backwardallow_unreachable=True)  # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [784, 512]], which is output 0 of TBackward, is at version 3; expected version 2 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

梯度计算所需的变量之一已被 inplace 操作修改。

bug分析

查看报错信息发现是loss.backward()部分报错，即在根据损失计算某个变量反向传播的梯度时，该变量被inplace操作修改了，即被原地修改，而不是将变化后的值赋给一个新的变量（类似于 a += 1 的操作）。因为反向传播遵循的是链式求导法则，这种原地修改的操作有时会使得模型计算梯度时找不到依赖的变量，从而导致梯度信息丢失或恶意篡改。

解决方法

找到网络模型中的 inplace 操作，将inplace=True改成 inplace=False，例如torch.nn.ReLU(inplace=False)
将代码中的“a+=b”之类的操作改为“c = a + b”
将loss.backward()函数内的参数retain_graph值设置为True, loss.backward(retain_graph=True)，如果retain_graph设置为False，计算过程中的中间变量使用完即被释放掉。

注意：所以大家在平常写代码的时候一定要养成良好的习惯，不乱用变量，注意代码的简洁性和高效性。

Pytorch Bug解决：RuntimeError:one of the variables needed for gradient computation has been modified相关推荐

PyTorch报错“RuntimeError: one of the variables needed for gradient computation has been modified by……”
1 问题描述今天在写作DeepLabV3+的代码时,遇到了一个问题, 程序报错: RuntimeError: one of the variables needed for gradient com ...
【Pytorch】RuntimeError: one of the variables needed for gradient computation has been modified by
0.前言最近在用Pytorch训练网络的时候出现了一个奇怪的错误,找了很久在偶然间试出了解决方法,因此记录一下,以作备忘.但具体原理还不是很清楚,也请各位大佬指教. 1.错误 RuntimeErro ...
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace
报错: RuntimeError: one of the variables needed for gradient computation has been modified by an inpla ...
排坑日记1：RuntimeError: one of the variables needed for gradient computation has been modified
问题描述在使用Pytorch复现DeepFill V1时,报如下错: RuntimeError: one of the variables needed for gradient computati ...
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace o
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace o ...
【完美解决】RuntimeError: one of the variables needed for gradient computation has been modified by an inp
正文在后面,往下拉即可~~~~~~~~~~~~ 欢迎各位深度学习的小伙伴订阅的我的专栏 Pytorch深度学习·理论篇+实战篇(2023版)专栏地址:
解决一个GAN训练过程中的报错：one of the variables needed for gradient computation has been modified by an inplace
跑一个GAN DEMO , 运行时出错. 出错代码: RuntimeError: one of the variables needed for gradient computation has be ...
one of the variables needed for gradient computation has been modified by an inplace operation
参考:https://www.cnblogs.com/liangzp/p/9207979.html 使用一个pytorch写的模型时,报错:RuntimeError:one of the variab ...
错误处理:one of the variables needed for gradient computation has been modified by inplace operation
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace o ...

Pytorch Bug解决：RuntimeError:one of the variables needed for gradient computation has been modified

Pytorch Bug解决：RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation

编程环境

Bug描述

bug分析

解决方法

Pytorch Bug解决：RuntimeError:one of the variables needed for gradient computation has been modified相关推荐

最新文章

热门文章