Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

文章目录

概
主要内容
- 深度
- 宽度
代码

Huang H., Wang Y., Erfani S., Gu Q., Bailey J. and Ma X. Exploring architectural ingredients of adversarially robust deep neural networks. In Advances in Neural Information Processing Systems (NIPS), 2021

概

本文是对现有的残差网络结构的探索, grid search一个鲁棒的结构.

主要内容

大家普遍认为越大的模型鲁棒性能会越好, 某种程度上如此, 但是现有的WRN(Wide ResNet)是为干净精度设计的, 对于鲁棒性并不是最优的.
现在的WRN有三个stage:

其越到后面越宽(即卷积核个数越多).
比如标准的WRN-34-10, 每个stage有5个block, 均乘上了factor=10.
本文便是探究block数量(即网络深度), 以及factor(即宽度)的影响.

深度

由上图可知, 削弱最后一个stage能够有效提升鲁棒性.

宽度

同样的, 削弱最后一个stage能够有效提升鲁棒性.

结合二者, 作者发现, 宽度比深度更有效, 维持10-10-4的比例的模型是最优的.
若进一步改为20-20-8(同比例scale), 鲁棒性接近饱和.

感觉给人的启示是, 最后一stage不能有太强的表达能力, 为什么?
我感觉还是残差连接的原因啊.

代码

原文代码

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks相关推荐

越线人群计数--Crossing-line Crowd Counting with Two-phase Deep Neural Networks
Crossing-line Crowd Counting with Two-phase Deep Neural Networks ECCV2016 人群计数有两种做法:1) region-of-int ...
Deep Neural Networks for YouTube Recommendations 双语译文+理解
目录 Deep Neural Networks for YouTube Recommendations Abstract 摘要 INTRODUCTION 1 引言 2 SYSTEM OVERVIEW ...
[论文阅读笔记58]Learning from Noisy Labels with Deep Neural Networks：A Survey
1.题目 Learning from Noisy Labels with Deep Neural Networks: A Survey 作者团队:韩国科学技术院(KAIST) Song H , Kim ...
Practical Advice for Building Deep Neural Networks
practical-advice-for-building-deep-neural-networks/ In our machine learning lab, we've accumulated t ...
Solving Inverse Problems With Deep Neural Networks
Solving Inverse Problems With Deep Neural Networks - Robustness Included 作者:Martin Genzel, Jan Macdo ...
3.Deep Neural Networks for YouTube Recommendations论文精细解读
一.总述今天分享的是Deep Neural Networks for Y ouTube Recommendations这篇论文的一些核心,这篇论文被称为推荐系统工程实践领域的一篇神文,每一个细节都值 ...
车牌检测识别--Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks
Towards End-to-End Car License Plates Detection and Recognition with Deep Neural Networks https://ar ...
批量残差网络-Aggregated Residual Transformations for Deep Neural Networks
Aggregated Residual Transformations for Deep Neural Networks Facebook AI Research 大牛 Ross Girshick K ...
DeepID3:Face Recognition with Very Deep Neural Networks
参考文献:<DeepID3:Face Recognition with Very Deep Neural Networks> 1 网络结构 2 特征提取 3 人脸验证
[译]深度神经网络的多任务学习概览(An Overview of Multi-task Learning in Deep Neural Networks)...
译自:http://sebastianruder.com/multi-task/ 1. 前言在机器学习中,我们通常关心优化某一特定指标,不管这个指标是一个标准值,还是企业KPI.为了达到这个目标,我 ...

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks

文章目录

概

主要内容

深度

宽度

代码

Exploring Architectural Ingredients of Adversarially Robust Deep Neural Networks相关推荐

最新文章

热门文章