模型量化--TBN:Convolutional Neural Network with Ternary Inputs and Binary Weights

TBN:Convolutional Neural Network with Ternary Inputs and Binary Weights
ECCV_2018 paper

TBN 用高效的 XOR, AND 及位运算代替传统CNN 中的算术运算
TBN replaces the arithmetical operations in standard CNNs with efficient XOR, AND and bitcount operations, and
thus provides an optimal tradeoff between memory, efficiency and performance.

provides ∼ 32× memory savings and 40× faster convolutional operations
TBN 的性能比 XNOR-Network 要高 5% 点左右

如果我们只对网络的权重参数进行二值化，那么得到的结果是内存空间降低32倍，计算速度提升2倍（因为避免了卷积中的乘法运算）
Binarizing the network weights can directly result in ∼ 32× memory saving over the real-valued counterparts, and meanwhile bring ∼ 2× computational efficiency by avoiding the multiplication operation in convolutions

如果我们对网络的权重和网络层的输入信号同时进行二值化，通过 XNOR and bitcount operations 代替卷积中的算术运算，那么可以提速 58倍。当然这么的问题就是性能下降较大。
On the other hand, simultaneously binarizing both weights and the input signals can result in 58× computational
efficiency by replacing the arithmetical operations in convolutions with XNOR and bitcount operations.

对此我们提出使用三值化网络层的输入信号来提升性能 ternary inputs constrain input signal values into −1, 0, and 1

Ternary-Binary Network (TBN) = 网络权重参数的二值化 + 网络层输入的三值化
By incorporating ternary layer-wise inputs with binary network weights, we propose a Ternary-Binary Network (TBN) that provides an optimal tradeoff between the performance and computational efficiency

TBN can provide ∼ 32× memory saving and 40× speedup over its real-valued CNN counterparts

下图显示不同量化策略对速度的提升影响

量化卷积过程

加速策略：
AND, XOR and bitcount operations

训练过程

分类性能对比

检测性能对比

模型量化--TBN:Convolutional Neural Network with Ternary Inputs and Binary Weights相关推荐

二值网络--TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights
TBN: Convolutional Neural Network with Ternary Inputs and Binary Weights ECCV2018 本文的思路就是: 对 weight ...
【读点论文】PP-LCNet: A Lightweight CPU Convolutional Neural Network，面向CPU级的技巧结合下的低延迟，较高精度模型
PP-LCNet: A Lightweight CPU Convolutional Neural Network 提出了一个基于MKLDNN加速策略的轻量级CPU网络,命名为PP-LCNet,它提高了 ...
模型加速--LCNN: Lookup-based Convolutional Neural Network
LCNN: Lookup-based Convolutional Neural Network CVPR2017 https://github.com/ildoonet/tf-lcnn https:/ ...
二值网络--Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy
Optimize Deep Convolutional Neural Network with Ternarized Weights and High Accuracy IEEE Winter Con ...
Synaptic Strength For Convolutional Neural Network
摘要卷积神经网络(Convolutional Neural network, CNNs)既需要大量的计算,又需要大量的内存,这阻碍了它在移动设备上的应用.受到神经科学文献中相关概念的启发,我们提出了 ...
A Convolutional Neural Network Model for Predicting a Product’s Function, Given Its Form
ABSTRACT 量化数字设计概念执行功能的能力目前需要使用昂贵的软件以及密集型解决方案,例如计算流体动力学.为了缓解这些挑战,这项工作的作者提出了一种基于3维卷积的深度学习方法,该方法可预测数字设计 ...
论文笔记 Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network
一.动机为了抽取文档级别的关系,许多方法使用远程监督(distant supervision )自动地生成文档级别的语料,从而用来训练关系抽取模型.最近也有很多多实例学习(multi-instanc ...
人群场景分析--Slicing Convolutional Neural Network for Crowd Video Understanding
Slicing Convolutional Neural Network for Crowd Video Understanding CVPR2016 http://www.ee.cuhk.edu.h ...
卷积神经网络Convolutional Neural Network (CNN)
卷积神经网络转载请注明:http://blog.csdn.net/stdcoutzyx/article/details/41596663 自今年七月份以来,一直在实验室负责卷积神经网络(Convol ...

模型量化--TBN:Convolutional Neural Network with Ternary Inputs and Binary Weights

模型量化--TBN:Convolutional Neural Network with Ternary Inputs and Binary Weights相关推荐

最新文章

热门文章