Deep Learning Toolkits 的比较(转)
Caffe Theano Torch TensorFlow
本文转自:https://github.com/zer0n/deepframeworks
Abstract. In this study, I evaluate some popular deep learning toolkits. The candidates are listed in alphabetical order: Caffe,CNTK, TensorFlow, Theano, and Torch. This is a dynamic document and the evaluation, to the best of my knowledge, is based on the current state of their code.
I also provide ratings in some areas because for a lot of people, ratings are useful. However, keep in mind that ratings are inherently subjective [1].
If you find something wrong or inadequate, please help improve by filing an issue.
Table of contents
- Modeling Capability
- Interfaces
- Model Deployment
- Performance
- Architecture
- Ecosystem
- Cross-platform
Modeling Capability
In this section, we evaluate each toolkit's ability to train common and state-of-the-art networks without writing too much code. Some of these networks are:
- ConvNets: AlexNet, OxfordNet, GoogleNet
- RecurrentNets: plain RNN, LSTM/GRU, bidirectional RNN
- Sequential modeling with attention.
In addition, we also evaluate the flexibility to create a new type of model.
Caffe
CNTK
As of today, CNTK is not usable for a variety of tasks such as sequence-2-sequence.
TensorFlow
- RNN API and implementation are suboptimal. The team also commented about it here and here.
- Bidirectional RNN not available yet
- No 3D convolution, which is useful for video recognition
Theano
Torch
- Excellent for conv nets. It's worth noting that temporal convolution can be done in TensorFlow/Theano via
conv2d
but that's a trick. The native interface for temporal convolution in Torch makes it slightly more intuitive to use. - Rich set of RNNs available through a non-official extension [2]
Left: graph model of CNTK/Theano/TensorFlow; Right: graph model of Caffe/Torch
Interfaces
Caffe
CNTK
TensorFlow
Theano
Torch
However, let's face it, Lua is not yet a mainstream language.
Model Deployment
How easy to deploy a new model?
Caffe
CNTK
TensorFlow
TF doesn't work on Windows yet so TF models can't be deployed on Windows devices though.
Theano
Torch
Performance
Single-GPU
All of these toolkits call cuDNN so as long as there’s no major computations or memory allocations at the outer level, they should perform similarly.
Soumith@FB has done some benchmarking for ConvNets. Deep Learning is not just about feedforward convnets, not just about ImageNet, and certainly not just about a few passes over the network. However, Soumith’s benchmark is the only notable one as of today. So we will base the Single-GPU performance rating based on his benchmark.
TensorFlow and Torch
Network | TF 0.6 [ref] | TF 0.8 [my run] | Torch FP32 [my run] |
---|---|---|---|
AlexNet | 292 | 97 | 81 |
Inception v1 | 1237 | 518 | 470 |
Theano
Multi-GPU
TBD
Architecture
Developer Zone
Caffe
Layer-wise design. The building block of a network in Caffe is layer.
- For new layer types, you have to define the full forward, backward, and gradient update. You can see an already long-list of layers implemented in (official) caffe.
- What's worse is that if you want to support both CPU and GPU, you need to implement extra functions, e.g.
Forward_gpu
andBackward_gpu
. - Worse, you need to assign an int id to your layer type and add that to the proto file. If your pull request is not merged early, you may need to change the id because someone else already claims that.
[Copied from my own answer on Quora]
CNTK
To be updated ...
TensorFlow
Theano
Torch
Torch7 and nn libraries are also well-designed with clean, modular interfaces.
Ecosystem
- Caffe and CNTK: C++
- TensorFlow: Python and C++
- Theano: Python
- Torch: Lua is not a mainstream language and hence libraries built for it are not as rich as ones built for Python.
Cross-platform
Caffe, CNTK, and Theano work on all OSes. TensorFlow and Torch do not work on Windows and there's no known plan to port from either camp.
Footnotes
[1] Note that I don’t aggregate ratings because different users/developers have different priorities.
[2] Disclaimer: I haven’t analyzed this extension carefully.
[3] See my blog post for why this is desirable.
Deep Learning Toolkits 的比较(转)相关推荐
- Evaluation of Deep Learning Toolkits
找不到其他的markdown编辑器了,写毕业论文用,先放着吧 原文地址:https://github.com/zer0n/deepframeworks Abstract In this study, ...
- (转) Deep Learning Resources
转自:http://www.jeremydjacksonphd.com/category/deep-learning/ Deep Learning Resources Posted on May 13 ...
- 几何深度学习(Geometric Deep Learning)技术
几何深度学习(Geometric Deep Learning)技术 几何深度学习综述 从论文Geometric Deep Learning: Grids, Groups, Graphs, Geodes ...
- 深度学习编译器综述The Deep Learning Compiler
深度学习编译器综述The Deep Learning Compiler The Deep Learning Compiler: A Comprehensive Survey 参考文献: https:/ ...
- 全文翻译(全文合集):TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
全文翻译(全文合集):TVM: An Automated End-to-End Optimizing Compiler for Deep Learning 摘要 人们越来越需要将机器学习应用到各种各样 ...
- 全文翻译(二): TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
全文翻译(二): TVM: An Automated End-to-End Optimizing Compiler for Deep Learning 3.优化计算图 计算图是在DL框架中表示程序的常 ...
- 全文翻译(一):TVM: An Automated End-to-End Optimizing Compiler for Deep Learning
全文翻译(一):TVM: An Automated End-to-End Optimizing Compiler for Deep Learning 摘要 人们越来越需要将机器学习应用到各种各样的硬件 ...
- TVM优化Deep Learning GPU算子
TVM优化Deep Learning GPU算子 高效的深度学习算子是深度学习系统的核心.通常,这些算子很难优化,需要HPC专家付出巨大的努力. 端到端张量IR / DSL堆栈TVM使这一过程变得更加 ...
- 深度学习编译与优化Deep Learning Compiler and Optimizer
深度学习编译与优化Deep Learning Compiler and Optimizer
最新文章
- cas 注销不关闭浏览器异常_一次浏览器请求的生命周期
- bzoj 4832 抵制克苏恩
- LDAPit's usage
- 【转】shell之for、while、until循环
- 带你读Paper丨分析ViT尚存问题和相对应的解决方案
- 安卓案例:列表控件上拉加载更多
- 5.3 恶意代码功能演示示例(上兴远程控制2014版)
- Pycharm中工程项目的多个python环境的管理使用方法的一点点理解
- Java实现OpenOffice将word转换为pdf
- 智能门锁电路图_智能门锁原理图,智能锁工作原理科普
- c语言calloc和malloc,使用malloc()、calloc()、free()和realloc()在C中进行动态内存分配
- 数据预处理部分的思维导图
- 10.cocos2d坐标系
- 自动化测试之Appium
- 江苏大学的计算机,毛启容-江苏大学计算机科学与通信工程学院
- 程序员久坐伤身,站起来,走出去,别回来
- 交互设计实用指南系列(11)—减少记忆负担
- 手机加速度传感器在Android横竖屏切换中的应用
- systemd和initd添加开机自启服务
- 欧姆龙cp1h-xa40dt-d与台达变频器modbus rtu通讯程序
热门文章
- mysql指数函数_SQL语言参考大全的目录
- [蓝桥杯][2017年第八届真题]小数第n位(数学)
- Python机器学习--KNN归一化、距离的惩罚
- 打印图形(2)(直角三角形)(C+Java)
- 把100减锐城1用计算机怎么算,北师大四年级下册数学
- 系统底部返回遮挡_一加7 Pro这个新功能让大家久等了:屏幕两侧一滑就可返回...
- 计算机编程语言的分类,解释型语言、编译型语言、脚本语言的区别
- Ubuntu14.04系统下安装配置OpenCV 4.0.0开发环境全过程
- C语言中open与fopen的的解释和区别
- ecos(redboot)移植剖析