Neural networks trained on large amounts of data have led to incredible technological leaps affecting nearly every part of our lives.

经过大量数据训练的神经网络导致了令人难以置信的技术飞跃，几乎影响了我们生活的每个部分。

These advances have come at a cost — namely the interpretability and explainability of data models. Corresponding with the complexity of the operation, the criteria for “choosing” a given output for an input becomes rather mysterious, leading some to refer to neural networks as a “black box” method.

这些进展是有代价的-即解释性和数据模型的explainability。 与操作的复杂性相对应，为输入“选择”给定输出的标准变得相当神秘，导致一些人将神经网络称为“黑匣子”方法。

Deep neural networks work so marvelously because they learn efficient representations of data, and they are intentionally constrained to capture complex, non-linear patterns in the data. The trade off of recognizing non-linear patterns is comparable to losing the sense of sight, only to gain a more subtle perception of sound. In the process of learning these representations, the features in each layer of a neural network change during training, and in different ways with different network architectures/datasets. This leads us to several research questions relevant to deep learning in general:

深度神经网络之所以如此出色，是因为它们学习了有效的数据表示形式，并且有意地被限制为捕获数据中的复杂非线性模式。 在识别非线性模式方面的权衡相当于失去了视觉感，只是获得了对声音的更微妙的感知 。 在学习这些表示的过程中，神经网络各层的特征在训练过程中会发生变化，并且会随着不同的网络体系结构/数据集以不同的方式发生变化。 这使我们提出了与深度学习相关的几个研究问题：

1. How can we understand the model performance in relationship to these changes?我们如何理解与这些变化相关的模型性能？
2. How do we identify the optimal number of units for each layer?我们如何确定每一层的最佳单位数？
3. How can we describe quantitatively the distribution of changes across a network?我们如何定量描述网络中变化的分布？

We attempted to answer these questions in our recent paper, “Feature Space Saturation during Training”, now available on arXiv. By applying Principal Component Analysis (PCA) to the learned representations in each layer during training, we can identify the layer size or number of dimensions needed to explain this variance, thus approximating the intrinsic dimensionality. With an approach similar to the Information Bottleneck method [1], SVCCA [2] and the Lottery Ticket Hypothesis [3], we attempt to identify the distribution of dynamics across the network over training.

我们尝试在arXiv上发表的最新论文“培训期间的特征空间饱和”中回答这些问题。 通过将主成分分析(PCA)应用于训练过程中每一层的学习表示，我们可以识别解释此差异所需的层大小或维数，从而近似固有维数。 通过类似于信息瓶颈方法[1]，SVCCA [2]和彩票假说[3]的方法，我们尝试通过训练来确定网络上的动态分布。

层本征空间的固有维数 (Intrinsic Dimensionality of Layer Eigenspaces)

To answer the question many dimensions are needed to explain the layer feature variance, we look at autoencoders.

为了回答这个问题，需要许多维度来解释图层特征差异，我们来看一下自动编码器。

Autoencoders learn a compact representation of the dataset. They are very useful for identifying the limits of neural network compression, as well as the dynamics of features/representations throughout model training.

自动编码器学习数据集的紧凑表示。 它们对于确定神经网络压缩的极限以及整个模型训练过程中特征/表示的动态非常有用。

The outputs of PCA are the various directions (eigenvectors) and eigenvalues corresponding to the correlation of the data. In this case, our input was the feature covariance matrix calculated on the layer representations throughout training. This matrix captures some dynamics relevant to the extent of feature independence and correlation. In other words, the extent to which some neurons respond in line with, or independent from other neurons in the layer. We call this projection the layer eigenspace.

PCA的输出是对应于数据相关性的各个方向(特征向量)和特征值。 在这种情况下，我们的输入是在整个训练过程中根据图层表示形式计算的特征协方差矩阵。 该矩阵捕获了一些与特征独立性和相关性有关的动力学。 换句话说， 某些神经元与该层中其他神经元一致或独立的React程度 。 我们称这种投影为层本征空间。

By selecting various thresholds for projecting the features (calculated by the cumulative sum of the eigenvalues), we were able to compare the reconstruction of images, in this case, a slice of pie. We call this threshold delta (δ), and it ranges from 0 to 100% of the directions needed to explain the variance. Thus, with a delta of 100%, we expect a near perfect reconstruction of the input — none are excluded.

通过选择各种阈值来投影特征(通过特征值的累加和计算)，我们能够比较图像的重建，在这种情况下是一片馅饼。 我们称此阈值增量(δ ) ，范围是解释方差所需方向的0到100％。 因此，在100％的变化量下，我们期望输入的重构接近完美-没有一个被排除。

In our experiment, beyond a threshold of 99%, a projection of the features onto the eigenvectors leaves an image which is only barely recognizable. Most of the structure is lost, indicating that the vast majority of feature subspace dimensions are needed for the model to perform. This approach allows us to compare feature spaces learned by a network and to understand the extent to which the network has learned optimal compressions of the data.

在我们的实验中，超过99％的阈值，特征在本征向量上的投影留下了几乎无法识别的图像。 大多数结构都丢失了，这表明模型需要绝大多数特征子空间尺寸。 这种方法使我们可以比较网络学习到的特征空间，并了解网络学习到数据最佳压缩的程度。

饱和度提供了进入模型训练的窗口 (Saturation Provides a Window into Model Training)

We call the proportion of eigenvectors needed to explain the variance of the layer features saturation. Each layer has a saturation index between 0 and 1 indicating the intrinsic dimensionality of the layer feature subspace. This allows us to compare saturation across layers within a deep neural network. Additionally, we compare the probe classifier approach by Alain and Bengio [4], which shows the relative ability of each layer’s output to perform the classification task.

我们称呼本征向量的比例来解释层特征饱和度的方差。 每个图层的饱和度索引介于0和1之间，表示图层特征子空间的固有维数。 这使我们能够比较深度神经网络中各层的饱和度。 另外，我们比较了Alain和Bengio [4]的探针分类器方法，该方法显示了每层输出执行分类任务的相对能力。

We observe that saturation reflects how the inference process is distributed. When the saturation is high, the layer’s features are changing in complex and non-linear ways, corresponding with a relatively high gain in probe classifier accuracy. Approaching the final layer, the marginal increase in layer accuracy decreases, as does the saturation in most cases. Thus, saturation is an indicator for optimal network depth, since redundant layers will have saturation converge towards zero in a tail pattern, described in the paper.

我们观察到饱和度反映了推理过程的分布方式。 当饱和度很高时，该层的特征将以复杂且非线性的方式变化，这对应于探针分类器精度的相对较高的增益。 接近最终层时，层精度的边际增加会降低，大多数情况下饱和度也会降低。 因此，饱和度是最佳网络深度的指标，因为冗余层的饱和度会在尾部模式中朝零收敛，如本文所述。

输入分辨率 (Input Resolution)

Input resolution is one of the 3 aspects to balance in a neural network architecture (including depth and width) [5].

输入分辨率是神经网络体系结构(包括深度和宽度)中要平衡的三个方面之一[5]。

Read more about it in the arXiv article or download the code used to generate the plots (delve Python library) on GitHub.

在arXiv文章中阅读有关它的更多信息，或在GitHub上下载用于生成绘图的代码(钻研Python库)。

Thanks to co-authors Mats L. Richter, Wolf Byttner, Anders Arpteg, and Mikael Huss. Many thanks for valuable feedback from Carl Thomé, Agrin Hilmkil, Rasmus Diederichsen, Richard Sieg, Alexis Drakopoulos, Saran N Subramaniyan, Piotr Migdał and Ulf Krumnack during writing of this article.

感谢合著者Mats L. Richter ，Wolf Byttner， Anders Arpteg和Mikael Huss 。 非常感谢卡尔· 汤姆(CarlThomé) ， 阿格琳·希尔姆基 ( Agrin Hilmkil) ，拉斯穆斯· 迪德里希森 (Rasmus Diederichsen)， 理查德·西格 ( Richard Sieg) ， 亚历克西斯·德拉科普洛斯 ( Alexis Drakopoulos) ， 萨兰·N · 苏布拉玛尼 扬 ， 皮奥特·米格达(PiotrMigdał)和乌尔夫·克鲁姆纳克 ( Ulf Krumnack)在本文撰写期间的宝贵反馈。

If you use this work in your research, please cite it as:

如果您在研究中使用这项工作，请引用为：

 @misc {shenk2020feature,  title={Feature Space Saturation during Training},  author={Justin Shenk and Mats L. Richter and Wolf Byttner and Anders Arpteg and Mikael Huss},  year={2020},  eprint={2006.08679},  archivePrefix={arXiv},  primaryClass={cs.LG}}