图像分类经典卷积神经网络—ResNet论文翻译(中英文对照版)—Deep Residual Learning for Image Recognition(深度残差学习的图像识别)
图像分类经典论文翻译汇总:[翻译汇总]
翻译pdf文件下载:[下载地址]
此版为中英文对照版,纯中文版请稳步:[ResNet纯中文版]
Deep Residual Learning for Image Recognition深度残差学习的图像识别 |
|||
Kaiming He(何恺明) | Xiangyu Zhang(张翔宇) | Shaoqing Ren(任少卿) | Jian Sun(孙剑) |
Microsoft Research(微软研究院){kahe, v-xiangz, v-shren, jiansun}@microsoft.com |
Abstract
摘要
1. Introduction
1. 引言
图1. 20层和56层的“简单”网络在CIFAR-10上的训练误差(左)和测试误差(右)。更深的网络有更高的训练误差和测试误差。如图4所示,ImageNet上具有类似现象。
Figure 2. Residual learning: a building block.
2. Related Work
2. 相关工作
3. Deep Residual Learning
3.1. Residual Learning
3. 深度残差学习
3.1. 残差学习
图7. CIFAR-10上层响应的标准差(std)。这些响应是每个3×3层的输出,在BN之后非线性之前。上图:以原始顺序显示层。下图:响应按降序排列。
3.2. Identity Mapping by Shortcuts
3.2. 快捷恒等映射
我们每隔几个堆叠层采用残差学习。构建块如图2所示。在本文中我们考虑构建块正式定义为:
方程(1)中x和F的维度必须是相等的。如果不是这种情况(例如,当更改输入/输出通道时),我们可以通过快捷连接执行线性投影Ws来匹配维度:
我们也可以使用方程(1)中的方阵Ws。但是我们将通过实验表明,恒等映射足以解决退化问题,并且是合算的,因此Ws仅在匹配维度时使用。
残差函数F的形式是可变的。本文中的实验包括有两层或三层(图5)的函数F,当然更多层也是可以的。但如果F只有一层,方程(1)类似于线性层:y = W1x + x,我们没有看到优势。
图5. ImageNet的深度残差函数F。左:图3中ResNet-34的构建块(在56×56的特征图上)。右:ResNet-50/101/152的“瓶颈”构建块。
我们还注意到,为了简单起见,尽管上述符号是关于全连接层的,但它们同样适用于卷积层。函数F(x, Wi)可以表示多个卷积层。元素加法在两个特征图上逐通道进行。
3.3. Network Architectures
3.3. 网络架构
我们测试了各种简单/残差网络,并观察到了一致的现象。为了提供讨论的实例,我们描述了ImageNet的两个模型如下。
表1. ImageNet架构。构建块显示在方括号中(同图5),以及构建块的堆叠数量。下采样通过步长为2的conv3_1, conv4_1和conv5_1执行。
值得注意的是我们的模型与VGG网络(图3左)相比,有更少的滤波器和更低的复杂度。我们的34层基准有36亿FLOP(乘加),仅是VGG-19(196亿FLOP)的18%。
3.4. Implementation
3.4. 实现
4. Experiments
4.1. ImageNet Classification
4. 实验
4.1. ImageNet分类
简单网络。我们首先评估18层和34层的简单网络。34层简单网络在图3(中间)。18层简单网络是一种类似的形式。有关详细的体系结构,请参见表1。
表2. ImageNet验证集上的Top-1错误率(%,10个裁剪图像进行测试)。相比于对应的简单网络,ResNet没有额外的参数。图4显示了训练过程。
Table 3. Error rates (%, 10-crop testing) on ImageNet validation. VGG-16 is based on our test. ResNet-50/101/152 are of option B that only uses projections for increasing dimensions.
我们认为这种优化难度不可能是由于梯度消失引起的。这些简单网络使用BN[16]训练,这保证了前向传播信号有非零方差。我们还验证了反向传播的梯度,结果显示其符合BN的正常标准。因此既不是前向信号消失也不是反向信号消失。实际上,34层简单网络仍能取得有竞争力的准确率(表3),这表明在某种程度上来说求解器仍有效。我们推测深度简单网络可能有指数级低收敛特性,这影响了训练误差的降低。这种优化困难的原因将来会研究。
表3. ImageNet验证集错误率(%,10个裁剪图像进行测试)。VGG16是基于我们的测试结果的。ResNet-50/101/152都是仅使用投影增加维度的B配置。
Residual Networks. Next we evaluate 18-layer and 34-layer residual nets (ResNets). The baseline architectures are the same as the above plain nets, expect that a shortcut connection is added to each pair of 3×3 filters as in Fig. 3 (right). In the first comparison (Table 2 and Fig. 4 right), we use identity mapping for all shortcuts and zero-padding for increasing dimensions (option A). So they have no extra parameter compared to the plain counterparts.
残差网络。接下来我们评估18层和34层残差网络(ResNets)。基准架构与上述的简单网络相同,如图3(右)所示,预计每对3×3滤波器都会添加快捷连接。在第一次比较(表2和图4右侧)中,我们对所有快捷连接都使用恒等映射和零填充以增加维度(选项A)。所以与对应的简单网络相比,它们没有额外的参数。
We have three major observations from Table 2 and Fig. 4. First, the situation is reversed with residual learning - the 34-layer ResNet is better than the 18-layer ResNet (by 2.8%). More importantly, the 34-layer ResNet exhibits considerably lower training error and is generalizable to the validation data. This indicates that the degradation problem is well addressed in this setting and we manage to obtain accuracy gains from increased depth.
我们从表2和图4中可以看到三个主要的观察结果。首先,通过残差学习这种情况发生的逆转——34层ResNet比18层ResNet更好(2.8%)。更重要的是,34层ResNet显示出相当低的训练误差,并且可以泛化到验证数据。这表明在这种情况下,退化问题得到了很好的解决,我们从增加的深度中设法获得了准确性收益。
Second, compared to its plain counterpart, the 34-layer ResNet reduces the top-1 error by 3.5% (Table 2), resulting from the successfully reduced training error (Fig. 4 right vs. left). This comparison verifies the effectiveness of residual learning on extremely deep systems.
第二,与对应的简单网络相比,由于成功的减少了训练误差,34层ResNet降低了3.5%的top-1错误率。这种比较证实了在极深系统中残差学习的有效性。
Last, we also note that the 18-layer plain/residual nets are comparably accurate (Table 2), but the 18-layer ResNet converges faster (Fig. 4 right vs. left). When the net is “not overly deep” (18 layers here), the current SGD solver is still able to find good solutions to the plain net. In this case, the ResNet eases the optimization by providing faster convergence at the early stage.
最后,我们还注意到18层的简单/残差网络同样地准确(表2),但18层ResNet收敛更快(图4右和左)。当网络“不过度深”时(18层),目前的SGD求解器仍能在简单网络中找到好的解。在这种情况下,ResNet通过在早期提供更快的收敛简便了优化。
Identity vs. Projection Shortcuts. We have shown that parameter-free, identity shortcuts help with training. Next we investigate projection shortcuts (Eqn.(2)). In Table 3 we compare three options: (A) zero-padding shortcuts are used for increasing dimensions, and all shortcuts are parameter-free (the same as Table 2 and Fig. 4 right); (B) projection shortcuts are used for increasing dimensions, and other shortcuts are identity; and (C) all shortcuts are projections.
恒等与投影快捷连接。我们已经表明没有参数、恒等快捷连接有助于训练。接下来我们探讨投影快捷连接(方程2)。在表3中我们比较了三个选项:(A)零填充快捷连接用来增加维度,所有的快捷连接是没有参数的(与表2和图4右相同);(B)投影快捷连接用来增加维度,其它的快捷连接是恒等的;(C)所有的快捷连接都是投影。
Table 3 shows that all three options are considerably better than the plain counterpart. B is slightly better than A. We argue that this is because the zero-padded dimensions in A indeed have no residual learning. C is marginally better than B, and we attribute this to the extra parameters introduced by many (thirteen) projection shortcuts. But the small differences among A/B/C indicate that projection shortcuts are not essential for addressing the degradation problem. So we do not use option C in the rest of this paper, to reduce memory/time complexity and model sizes. Identity shortcuts are particularly important for not increasing the complexity of the bottleneck architectures that are introduced below.
表3显示,所有三个选项都比对应的简单网络好很多。选项B比A略好。我们认为这是因为A中的零填充确实没有残差学习。选项C比B稍好,我们把这归因于许多(十三)投影快捷连接引入了额外参数。但A/B/C之间的细微差异表明,投影快捷连接对于解决退化问题不是至关重要的。因为我们在本文的剩余部分不再使用选项C,以减少内存/时间复杂性和模型大小。恒等快捷连接对于不增加下面介绍的瓶颈结构的复杂性尤为重要。
Deeper Bottleneck Architectures. Next we describe our deeper nets for ImageNet. Because of concerns on the training time that we can afford, we modify the building block as a bottleneck design. For each residual function F, we use a stack of 3 layers instead of 2 (Fig. 5). The three layers are 1×1, 3×3, and 1×1 convolutions, where the 1×1 layers are responsible for reducing and then increasing (restoring) dimensions, leaving the 3×3 layer a bottleneck with smaller input/output dimensions. Fig. 5 shows an example, where both designs have similar time complexity.
更深的瓶颈结构。接下来我们描述ImageNet中我们使用的更深的网络。由于考虑到我们能承受的训练时间,我们将构建块修改为瓶颈设计。对于每个残差函数F,我们使用3层堆叠而不是2层(图5)。这三层是1×1,3×3和1×1卷积,其中1×1层负责减小然后增加(恢复)维度,使3×3层成为具有较小输入/输出维度的瓶颈。图5展示了一个示例,两个设计具有相似的时间复杂度。
The parameter-free identity shortcuts are particularly important for the bottleneck architectures. If the identity shortcut in Fig. 5 (right) is replaced with projection, one can show that the time complexity and model size are doubled, as the shortcut is connected to the two high-dimensional ends. So identity shortcuts lead to more efficient models for the bottleneck designs.
无参数恒等快捷连接对于瓶颈架构尤为重要。如果图5(右)中的恒等快捷连接被投影替换,则可以显示出时间复杂度和模型大小加倍,因为快捷连接是连接到两个高维端。因此,恒等快捷连接可以为瓶颈设计得到更有效的模型。
50-layer ResNet: We replace each 2-layer block in the 34-layer net with this 3-layer bottleneck block, resulting in a 50-layer ResNet (Table 1). We use option B for increasing dimensions. This model has 3.8 billion FLOPs.
50层ResNet:我们用3层瓶颈块替换34层网络中的每一个2层块,得到了一个50层ResNet(表1)。我们使用选项B来增加维度。该模型有38亿FLOP。
101-layer and 152-layer ResNet: We construct 101-layer and 152-layer ResNets by using more 3-layer blocks (Table 1). Remarkably, although the depth is significantly increased, the 152-layer ResNet (11.3 billion FLOPs) still has lower complexity than VGG-16/19 nets (15.3/19.6 billion FLOPs).
101层和152层ResNet:我们通过使用更多的3层瓶颈块来构建101层和152层ResNets(表1)。值得注意的是,尽管深度显著增加,但152层ResNet(113亿FLOP)仍然比VGG-16/19网络(153/196亿FLOP)具有更低的复杂度。
The 50/101/152-layer ResNets are more accurate than the 34-layer ones by considerable margins (Table 3 and 4). We do not observe the degradation problem and thus enjoy significant accuracy gains from considerably increased depth. The benefits of depth are witnessed for all evaluation metrics (Table 3 and 4).
50/101/152层ResNet比34层ResNet的准确性要高得多(表3和4)。我们没有观察到退化问题,因此可以从显著增加的深度中获得显著的准确性收益。所有评估指标都能证明深度的收益(表3和表4)。
Comparisons with State-of-the-art Methods. In Table 4 we compare with the previous best single-model results. Our baseline 34-layer ResNets have achieved very competitive accuracy. Our 152-layer ResNet has a single-model top-5 validation error of 4.49%. This single-model result outperforms all previous ensemble results (Table 5). We combine six models of different depth to form an ensemble (only with two 152-layer ones at the time of submitting). This leads to 3.57% top-5 error on the test set (Table 5). This entry won the 1st place in ILSVRC 2015.
Table 4. Error rates (%) of single-model results on the ImageNet validation set (except † reported on the test set).
Table 5. Error rates (%) of ensembles. The top-5 error is on the test set of ImageNet and reported by the test server.
与最先进的方法比较。在表4中,我们与以前最好的单一模型结果进行比较。我们基准的34层ResNet已经取得了非常有竞争力的准确性。我们的152层ResNet单模型具有4.49%的top-5错误率。这个单一模型的结果胜过以前的所有集成结果(表5)。我们结合了六种不同深度的模型,形成一个集成模型(在提交时仅有两个152层)。这在测试集上得到了3.5%的top-5错误率(表5)。这次提交在2015年ILSVRC中荣获了第一名。
表4. 单一模型在ImageNet验证集上的错误率(%)(除了†是测试集上报告的错误率)。
表5. 集成模型的错误率(%)。top-5错误率是ImageNet测试集上由测试服务器报告的。
4.2. CIFAR-10 and Analysis
We conducted more studies on the CIFAR-10 dataset [20], which consists of 50k training images and 10k testing images in 10 classes. We present experiments trained on the training set and evaluated on the test set. Our focus is on the behaviors of extremely deep networks, but not on pushing the state-of-the-art results, so we intentionally use simple architectures as follows.
4.2. CIFAR-10和分析
我们对CIFAR-10数据集[20]进行了更多的研究,其中包括10个类别中的5万张训练图像和1万张测试图像。我们介绍了在训练集上进行训练和在测试集上进行评估的实验。我们的焦点在于极深网络的行为,而不是产生最先进的结果,所以我们有意使用如下的简单架构。
The plain/residual architectures follow the form in Fig. 3 (middle/right). The network inputs are 32×32 images, with the per-pixel mean subtracted. The first layer is 3×3 convolutions. Then we use a stack of 6n layers with 3×3 convolutions on the feature maps of sizes {32, 16, 8} respectively, with 2n layers for each feature map size. The numbers of filters are {16, 32, 64} respectively. The subsampling is performed by convolutions with a stride of 2. The network ends with a global average pooling, a 10-way fully-connected layer, and softmax. There are totally 6n+2 stacked weighted layers. The following table summarizes the architecture:
When shortcut connections are used, they are connected to the pairs of 3×3 layers (totally 3n shortcuts). On this dataset we use identity shortcuts in all cases (i.e., option A), so our residual models have exactly the same depth, width, and number of parameters as the plain counterparts.
简单/残差架构如图3(中/右)的形式。网络输入是32×32的图像,每个像素减去均值。第一层是3×3卷积。然后我们在大小为{32,16,8}的特征图上分别使用了带有3×3卷积的6n个堆叠层,每个特征图大小使用2n层。滤波器数量分别为{16,32,64}。下采样由步长为2的卷积进行。网络以全局平均池化,一个10维全连接层和softmax作为结束。共有6n+2个堆叠的加权层。下表总结了这个架构:
当使用快捷连接时,它们连接到成对的3×3卷积层上(共3n个快捷连接)。在这个数据集上,我们在所有案例中都使用恒等快捷连接(即选项A),因此我们的残差模型与对应的简单模型具有完全相同的深度,宽度和参数数量。
We use a weight decay of 0.0001 and momentum of 0.9, and adopt the weight initialization in [12] and BN [16] but with no dropout. These models are trained with a mini-batch size of 128 on two GPUs. We start with a learning rate of 0.1, divide it by 10 at 32k and 48k iterations, and terminate training at 64k iterations, which is determined on a 45k/5k train/val split. We follow the simple data augmentation in [24] for training: 4 pixels are padded on each side, and a 32×32 crop is randomly sampled from the padded image or its horizontal flip. For testing, we only evaluate the single view of the original 32×32 image.
我们使用的权重衰减为0.0001和动量为0.9,并采用[12]和BN[16]中的权重初始化,但没有使用dropout。这些模型在两个GPU上进行训练,批处理大小为128。我们开始使用的学习率为0.1,在32k次和48k次迭代后学习率除以10,并在64k次迭代后终止训练,这是由45k/5k的训练/验证集分割决定的。我们按照[24]中的简单数据增强进行训练:每边填充4个像素,并从填充图像或其水平翻转图像中随机采样32×32的裁剪图像。对于测试,我们只评估原始32×32图像的单一视图。
We compare n = {3,5,7,9}, leading to 20, 32, 44, and 56-layer networks. Fig. 6 (left) shows the behaviors of the plain nets. The deep plain nets suffer from increased depth, and exhibit higher training error when going deeper. This phenomenon is similar to that on ImageNet (Fig. 4, left) and on MNIST (see [42]), suggesting that such an optimization difficulty is a fundamental problem.
Figure 6. Training on CIFAR-10. Dashed lines denote training error, and bold lines denote testing error. Left: plain networks. The error of plain-110 is higher than 60% and not displayed. Middle: ResNets. Right: ResNets with 110 and 1202 layers.
我们比较了n = {3,5,7,9},得到了20层,32层,44层和56层的网络。图6(左)显示了简单网络的行为。深度简单网络经历了深度增加,随着深度增加表现出了更高的训练误差。这种现象类似于ImageNet中(图4,左)和MNIST中(请看[42])的现象,表明这种优化困难是一个基本的问题。
图6. 在CIFAR-10上训练。虚线表示训练误差,粗线表示测试误差。左:简单网络。简单的110层网络错误率超过60%,没有展示。中:ResNet。右:110层和1202层的ResNet。
Fig. 6 (middle) shows the behaviors of ResNets. Also similar to the ImageNet cases (Fig. 4, right), our ResNets manage to overcome the optimization difficulty and demonstrate accuracy gains when the depth increases.
图6(中)显示了ResNet的行为。这也与ImageNet的情况类似(图4,右),我们的ResNet设法克服优化困难,并随着深度的增加呈现了准确性收益。
We further explore n = 18 that leads to a 110-layer ResNet. In this case, we find that the initial learning rate of 0.1 is slightly too large to start converging. So we use 0.01 to warm up the training until the training error is below 80% (about 400 iterations), and then go back to 0.1 and continue training. The rest of the learning schedule is as done previously. This 110-layer network converges well (Fig. 6, middle). It has fewer parameters than other deep and thin networks such as FitNet [35] and Highway [42] (Table 6), yet is among the state-of-the-art results (6.43%, Table 6).
Table 6. Classification error on the CIFAR-10 test set. All methods are with data augmentation. For ResNet-110, we run it 5 times and show “best (mean±std)” as in [43].
我们进一步探索了n = 18得到了110层的ResNet。在这种情况下,我们发现0.1的初始学习率对于收敛来说太大了。因此我们使用0.01的学习率开始训练,直到训练误差低于80%(大约400次迭代),然后学习率变回到0.1并继续训练。学习过程的剩余部分与前面做的一样。这个110层网络收敛的很好(图6,中)。它与其它的深且窄的网络例如FitNet[35]和Highway[42]相比有更少的参数,然而结果仍在目前最好的结果中(6.43%,表6)。
表6. 在CIFAR-10测试集上的分类误差。所有的方法都使用了数据增强。对于ResNet-110,像论文[43]中那样,我们运行了5次并展示了“best (mean±std)”。
Analysis of Layer Responses. Fig. 7 shows the standard deviations (std) of the layer responses. The responses are the outputs of each 3×3 layer, after BN and before other nonlinearity (ReLU/addition). For ResNets, this analysis reveals the response strength of the residual functions. Fig. 7 shows that ResNets have generally smaller responses than their plain counterparts. These results support our basic motivation (Sec.3.1) that the residual functions might be generally closer to zero than the non-residual functions. We also notice that the deeper ResNet has smaller magnitudes of responses, as evidenced by the comparisons among ResNet-20, 56, and 110 in Fig. 7. When there are more layers, an individual layer of ResNets tends to modify the signal less.
层响应分析。图7显示了层响应的标准偏差(std)。这些响应每个3×3层的输出,在BN之后和其他非线性(ReLU/加法)之前。对于ResNets,该分析揭示了残差函数的响应强度。图7显示ResNet的响应比其对应的简单网络的响应更小。这些结果支持了我们的基本动机(第3.1节),即残差函数通常比非残差函数更接近零。通过比较图7中ResNet-20,ResNet-56和ResNet-110,我们还注意到,更深的ResNet具有较小的响应幅度。当层数更多时,单层ResNet趋向于更少地修改信号。
Exploring Over 1000 layers. We explore an aggressively deep model of over 1000 layers. We set n = 200 that leads to a 1202-layer network, which is trained as described above. Our method shows no optimization difficulty, and this 103-layer network is able to achieve training error <0.1% (Fig. 6, right). Its test error is still fairly good (7.93%, Table 6).
探索超过1000层。我们探索了一个超过1000层非常深的模型。我们设置n = 200,得到了1202层的网络,其训练如上所述。我们的方法显示没有优化困难,这个103层的网络能够实现训练误差<0.1%(图6,右图)。其测试误差仍然很好(7.93%,表6)。
But there are still open problems on such aggressively deep models. The testing result of this 1202-layer network is worse than that of our 110-layer network, although both have similar training error. We argue that this is because of overfitting. The 1202-layer network may be unnecessarily large (19.4M) for this small dataset. Strong regularization such as maxout [9] or dropout [13] is applied to obtain the best results ([9, 25, 24, 34]) on this dataset. In this paper, we use no maxout/dropout and just simply impose regularization via deep and thin architectures by design, without distracting from the focus on the difficulties of optimization. But combining with stronger regularization may improve results, which we will study in the future.
但是,这种极深的模型仍然存在着一些未解决的问题。这个1202层网络的测试结果比我们的110层网络的测试结果更差,虽然两者都具有类似的训练误差。我们认为这是过拟合造成的。对于这种小型数据集,1202层网络可能过大(19.4M)。在这个数据集应用强大的正则化,如maxout[9]或者dropout[13]来获得最佳结果([9,25,24,34])。在本文中,我们不使用maxout/dropout,只是简单地通过设计深且窄的架构简单地进行正则化,而不会分散集中在优化难点上的注意力。但结合更强的正规化可能会改善结果,我们将来会做研究。
4.3. Object Detection on PASCAL and MS COCO
Our method has good generalization performance on other recognition tasks. Table 7 and 8 show the object detection baseline results on PASCAL VOC 2007 and 2012 [5] and COCO [26]. We adopt Faster R-CNN [32] as the detection method. Here we are interested in the improvements of replacing VGG-16 [40] with ResNet-101. The detection implementation (see appendix) of using both models is the same, so the gains can only be attributed to better networks. Most remarkably, on the challenging COCO dataset we obtain a 6.0% increase in COCO’s standard metric (mAP@[.5, .95]), which is a 28% relative improvement. This gain is solely due to the learned representations.
Table 7. Object detection mAP (%) on the PASCAL VOC 2007/2012 test sets using baseline Faster R-CNN. See also appendix for better results.
Table 8. Object detection mAP (%) on the COCO validation set using baseline Faster R-CNN. See also appendix for better results.
4.3. 在PASCAL和MS COCO上的目标检测
我们的方法对其他识别任务有很好的泛化性能。表7和表8显示了PASCAL VOC 2007和2012[5]以及COCO[26]的目标检测基准结果。我们采用Faster R-CNN[32]作为检测方法。在这里,我们感兴趣的是用ResNet-101替换VGG-16[40]。这两种模型的检测方法(见附录)是一样的,所以收益只能归因于更好的网络。最显著的是,在有挑战性的COCO数据集中,COCO的标准度量指标(mAP@[.5,.95])增长了6.0%,相对改善了28%。这种收益完全是由于学习到的表示。
表7. 在PASCAL VOC 2007/2012测试集上使用基准Faster R-CNN的目标检测mAP(%)。更好的结果请看附录。
表8. 在COCO验证集上使用基准Faster R-CNN的目标检测mAP(%)。更好的结果请看附录。
Based on deep residual nets, we won the 1st places in several tracks in ILSVRC & COCO 2015 competitions: ImageNet detection, ImageNet localization, COCO detection, and COCO segmentation. The details are in the appendix.
基于深度残差网络,我们在ILSVRC和COCO 2015竞赛的几个任务中获得了第一名,分别是:ImageNet检测,ImageNet定位,COCO检测,COCO分割。详见附录。
References
参考文献
[1] Y. Bengio, P. Simard, and P. Frasconi. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2):157–166, 1994.
[2] C. M. Bishop. Neural networks for pattern recognition. Oxford university press, 1995.
[3] W. L. Briggs, S. F. McCormick, et al. A Multigrid Tutorial. Siam, 2000.
[4] K. Chatfield, V. Lempitsky, A. Vedaldi, and A. Zisserman. The devil is in the details: an evaluation of recent feature encoding methods. In BMVC, 2011.
[5] M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The Pascal Visual Object Classes (VOC) Challenge. IJCV, pages 303–338, 2010.
[6] S. Gidaris and N. Komodakis. Object detection via a multi-region & semantic segmentation-aware cnn model. In ICCV, 2015.
[7] R. Girshick. Fast R-CNN. In ICCV, 2015.
[8] R. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, 2014.
[9] X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In AISTATS, 2010.
[10] I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio. Maxout networks. arXiv:1302.4389, 2013.
[11] K. He and J. Sun. Convolutional neural networks at constrained time cost. In CVPR, 2015.
[12] K. He, X. Zhang, S. Ren, and J. Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. In ECCV, 2014.
[13] K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In ICCV, 2015.
[14] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing coadaptation of feature detectors. arXiv:1207.0580, 2012.
[15] S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
[16] S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, 2015.
[17] H. Jegou, M. Douze, and C. Schmid. Product quantization for nearest neighbor search. TPAMI, 33, 2011.
[18] H. Jegou, F. Perronnin, M. Douze, J. Sanchez, P. Perez, and C. Schmid. Aggregating local image descriptors into compact codes. TPAMI, 2012.
[19] Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. arXiv:1408.5093, 2014.
[20] A. Krizhevsky. Learning multiple layers of features from tiny images. Tech Report, 2009.
[21] A. Krizhevsky, I. Sutskever, and G. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
[22] Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1989.
[23] Y. LeCun, L. Bottou, G. B. Orr, and K.-R.M¨uller. Efficient backprop. In Neural Networks: Tricks of the Trade, pages 9–50. Springer, 1998.
[24] C.-Y. Lee, S. Xie, P. Gallagher, Z. Zhang, and Z. Tu. Deeplysupervised nets. arXiv:1409.5185, 2014.
[25] M. Lin, Q. Chen, and S. Yan. Network in network. arXiv:1312.4400, 2013.
[26] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll´ar, and C. L. Zitnick. Microsoft COCO: Common objects in context. In ECCV. 2014.
[27] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR, 2015.
[28] G. Mont´ufar, R. Pascanu, K. Cho, and Y. Bengio. On the number of linear regions of deep neural networks. In NIPS, 2014.
[29] V. Nair and G. E. Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, 2010.
[30] F. Perronnin and C. Dance. Fisher kernels on visual vocabularies for image categorization. In CVPR, 2007.
[31] T. Raiko, H. Valpola, and Y. LeCun. Deep learning made easier by linear transformations in perceptrons. In AISTATS, 2012.
[32] S. Ren, K. He, R. Girshick, and J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In NIPS, 2015.
[33] S. Ren, K. He, R. Girshick, X. Zhang, and J. Sun. Object detection networks on convolutional feature maps. arXiv:1504.06066, 2015.
[34] B. D. Ripley. Pattern recognition and neural networks. Cambridge university press, 1996.
[35] A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio. Fitnets: Hints for thin deep nets. In ICLR, 2015.
[36] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. arXiv:1409.0575, 2014.
[37] A. M. Saxe, J. L. McClelland, and S. Ganguli. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks. arXiv:1312.6120, 2013.
[38] N. N. Schraudolph. Accelerated gradient descent by factor-centering decomposition. Technical report, 1998.
[39] N. N. Schraudolph. Centering neural network gradient factors. In Neural Networks: Tricks of the Trade, pages 207–226. Springer, 1998.
[40] P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. Le-Cun. Overfeat: Integrated recognition, localization and detection using convolutional networks. In ICLR, 2014.
[41] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015.
[42] R. K. Srivastava, K. Greff, and J. Schmidhuber. Highway networks. arXiv:1505.00387, 2015.
[43] R. K. Srivastava, K. Greff, and J. Schmidhuber. Training very deep networks. 1507.06228, 2015.
[44] C. Szegedy,W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In CVPR, 2015.
[45] R. Szeliski. Fast surface interpolation using hierarchical basis functions. TPAMI, 1990.
[46] R. Szeliski. Locally adapted hierarchical basis preconditioning. In SIGGRAPH, 2006.
[47] T. Vatanen, T. Raiko, H. Valpola, and Y. LeCun. Pushing stochastic gradient towards second-order methods–backpropagation learning with transformations in nonlinearities. In Neural Information Processing, 2013.
[48] A. Vedaldi and B. Fulkerson. VLFeat: An open and portable library of computer vision algorithms, 2008.
[49] W. Venables and B. Ripley. Modern applied statistics with s-plus. 1999.
[50] M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional neural networks. In ECCV, 2014.
[1] We have experimented with more training iterations (3×) and still observed the degradation problem, suggesting that this problem cannot be feasibly addressed by simply using more iterations.
图像分类经典卷积神经网络—ResNet论文翻译(中英文对照版)—Deep Residual Learning for Image Recognition(深度残差学习的图像识别)相关推荐
- 图像分类经典卷积神经网络—ResNet论文翻译(纯中文版)—Deep Residual Learning for Image Recognition(深度残差学习的图像识别)
图像分类经典论文翻译汇总:[翻译汇总] 翻译pdf文件下载:[下载地址] 此版为纯中文版,中英文对照版请稳步:[ResNet中英文对照版] Deep Residual Learning for Ima ...
- 图像分类经典卷积神经网络—GoogLeNet论文翻译(中英文对照版)—Going Deeper with Convolutions(走向更深的卷积神经网络)
图像分类经典论文翻译汇总:[翻译汇总] 翻译pdf文件下载:[下载地址] 此版为中英文对照版,纯中文版请稳步:[GoogLeNet纯中文版] Going Deeper with Convolution ...
- 图像分类经典卷积神经网络—ZFNet论文翻译(中英文对照版)—Visualizing and Understanding Convolutional Networks(可视化和理解卷积网络)
图像分类经典论文翻译汇总:[翻译汇总] 翻译pdf文件下载:[下载地址] 此版为中英文对照版,纯中文版请稳步:[ZFNet纯中文版] Visualizing and Understanding Con ...
- 图像分类经典卷积神经网络—GoogLeNet论文翻译(纯中文版)—Going Deeper with Convolutions(走向更深的卷积神经网络)
图像分类经典论文翻译汇总:[翻译汇总] 翻译pdf文件下载:[下载地址] 此版为纯中文版,中英文对照版请稳步:[GoogLeNet纯中英文对照版] Going Deeper with Convolut ...
- ResNet论文翻译——中英文对照
文章作者:Tyan 博客:noahsnail.com | CSDN | 简书 声明:作者翻译论文仅为学习,如有侵权请联系作者删除博文,谢谢! 翻译论文汇总:https://github.com ...
- ResNet论文翻译——中英文对照+标注总结
Deep Residual Learning for Image Recognition 文章目录 Deep Residual Learning for Image Recognition Abstr ...
- 卷积神经网络——FPN论文翻译
论文链接: https://arxiv.org/abs/1612.03144 Feature Pyramid Networks for Object Detection 0.摘要 特征金字塔是识别系统 ...
- 卷积神经网络——YOLOv3论文翻译
论文链接:https://arxiv.org/abs/1804.02767 YOLOv3: An Incremental Improvement Abstract (摘要) 我们提出了一些对YOLO的 ...
- Deep Residual Learning for Image Recognition(ResNet)论文翻译及学习笔记
[论文翻译]:Deep Residual Learning for Image Recognition [论文来源]:Deep Residual Learning for Image Recognit ...
最新文章
- apache 访问出现403 Forbidden
- 为什么越来越多的手机浏览器转型新闻客户端?
- 在学习Python中,这个知识我们一定要看一遍,记不住没关系单一定要知道,字符串常用函数用法
- maven 打包指定依赖包_[Maven]-Maven基础-01-基础概念
- 主机与虚拟机桥接出现ping不通解决方案
- OCS2007R2升级LyncSrv2013 PART1:基础准备
- php 打印 wap,PHP 输出简单动态WAP页面
- java mysql 单例模式_java 单例模式(singleton)连接 mysql 数据库
- 『TensorFlow Internals』笔记_系统架构
- html返回底部代码,返回页面顶部及去页面底部的js实现代码
- 【CDN学习笔记5】源站IP变更后导致图片显示不出来的案例
- 2010总结,2011展望
- android视频动态壁纸app,短视频动态壁纸转换
- 10大渗透神器,渗透测试必备
- YOLOv3使用笔记
- 信捷plc c 语言全局变量,三菱PLC编程中,跳转指令CJ、子程序调用CALL和中断指令有什么区别?...
- 搞定调度算法只需三分钟
- Maven基础简介+安装说明
- 开源Android 图库
- [STM32F103C8T6] 超声波测距
热门文章
- 西门子plc博图与优傲UR机器人进行Profinet通讯,s7-1200/1500 与UR机器人通讯,实际应用案例使用中
- uniapp onReachBottom 不触发
- ​如何通过微信打新债?
- 怀旧服为什么显示服务器不兼容,《魔兽世界怀旧服》加载出错怎么办 加载出错解决方案...
- linux 下的lsb_release -a 命令查询系统版本
- 报错:实体名称必须紧跟在 '' 后面
- 12.Opencv大作业——实现鼠标绘制基本图形
- 利用LaTeX写硕士论文历程之安装配置环境
- python数据质量检查
- DBeaver导出结果集为CSV文件,数据用引号括起来