nn.Identity() 是个函数，不是矩阵！
恒等函数 f ( x ) = x f(x)=x f(x)=x
罢了

说明

nn.Identity will just return its input, but doesn’t show that it’s a view.
If you thus manipulate the input inplace, the output of nn.Identity will also be changed:

a = torch.arange(4.)
m = nn.Identity()
input_identity = m(a)print(a)
> tensor([0., 1., 2., 3.])print(input_identity)
> tensor([0., 1., 2., 3.])

manipulate inplace

a[0] = 2.
print(a)
> tensor([2., 1., 2., 3.])print(input_identity)
> tensor([2., 1., 2., 3.])

As mentioned before, nn.Identity will just return the input without any clone usage or manipulation of the input. The input and output would thus be the same.

Yes, this “pass-through” layer can easily be written manually, which was also the reason feature requests were declined in the past.
However, since a lot of users were rewriting the same layer to e.g. replace specific modules inside a larger model, this layer was introduced.

You might not want to use this layer as it’s not “doing anything” besides just returning the input.
However, there are use cases where users needed exactly this (e.g. to replace another layer) and were manually creating custom modules to do so and asked for the nn.Identity layer in the PyTorch nn backend. Since more and more users were depending on it, it was created.
However, as already said, this layer might not be interesting for you.

skip connection

Skip connections would usually a add activations instead of concatenating them as seen in the resnet example 12.

An often used use case for nn.Identity would be to get the “features” of a pretrained model instead of the class logits.
最后一层本来是用作分类的，用 softmax函数或者 fully connected 函数，但是用 nn.identtiy() 函数把最后一层替换掉，相当于得到分类之前的特征！
在迁移学习里有用。

Here is an example:

model = models.resnet18()
# replace last linar layer with nn.Identity
model.fc = nn.Identity()# get features for input
x = torch.randn(1, 3, 224, 224)
out = model(x)
print(out.shape)
> torch.Size([1, 512])

来自

What is nn.Identity() used for?

nn.Identity()相关推荐

torch.nn.identity()方法详解
先看代码 m = nn.Identity( 54, unused_argument1=0.1, unused_argument2=False )input = torch.randn(128, 20) ...
【pytorch】torch.nn.Identity()
identity模块不改变输入,直接return input 一种编码技巧吧,比如我们要加深网络,有些层是不改变输入数据的维度的, 在增减网络的过程中我们就可以用identity占个位置,这样网络整体 ...
torch.nn、(二)
参考 torch.nn.(二) - 云+社区 - 腾讯云目录 Recurrent layers RNN LSTM GRU RNNCell LSTMCell GRUCell Transformer l ...
torch.nn.Linear详解
在学习transformer时,遇到过非常频繁的nn.Linear()函数,这里对nn.Linear进行一个详解. 参考:https://pytorch.org/docs/stable/_module ...
python batchnorm2d_Python nn.BatchNorm2d方法代碼示例
本文整理匯總了Python中torch.nn.BatchNorm2d方法的典型用法代碼示例.如果您正苦於以下問題:Python nn.BatchNorm2d方法的具體用法?Python nn.Batc ...
真香！Vision Transformer 快速实现 Mnist 识别
作者 | 李秋键出品 | AI科技大本营(ID:rgznai100) 引言:基于深度学习的方法在计算机视觉领域中最典型的应用就是卷积神经网络CNN.CNN中的数据表示方式是分层的,高层特征表示依赖于 ...
紧跟大佬的步伐：关于我亲自动手复现了恺明新作MAE这件事
点击上方"视学算法",选择加"星标"或"置顶" 重磅干货,第一时间送达作者丨CW不要無聊的風格@知乎(已授权) 来源丨https://zh ...
13个你一定要知道的PyTorch特性
点击上方"视学算法",选择加"星标"或"置顶" 重磅干货,第一时间送达编译 | ronghuaiyang 来源 | 人工智能前沿讲习编辑 ...
CNN、Transformer、MLP架构的经验性分析
点击上方"3D视觉工坊",选择"星标" 干货第一时间送达作者丨pprp 来源丨GiantPandaCV 编辑丨极市平台导读 ViT的兴起挑战了CNN的地位, ...

nn.Identity()

说明

manipulate inplace

skip connection

来自

nn.Identity()相关推荐

最新文章

热门文章