Ubuntu和Centos安装Tensorflow教程PyTorch
更多请关注:
- https://github.com/chanhal
- https://www.zhihu.com/people/chanhal
- https://github.com/chanhal/config_ml_env (原版PDF下载)
- chanhal@outlook.com
文字版
Ubuntu和Centos安装Tensorflow教程&TyTorch
Ubuntu安装TensorFlow
安装Ubuntu 16.04 LTS
下载地址:http://releases.ubuntu.com/16.04/
[外链图片转存失败(img-SQjaHmqR-1568858900081)(media/3e7357d001043b9d534ae2d8760315d2.png)]
Ubuntu
ISO下载完成之后利用ultraiso写入U盘,制作系统启动盘,将系统设置为U盘启动,机器重启后就开始安装系统(DELL台式机,F12可以设置U盘启动)。制作启动U盘时有两点需要特别注意:
U盘接口最好需要是2.0,对于3.0的接口有的电脑主板不一定原生支持
对于新安装硬盘(例如新买电脑),尤其是SSD,安装前一定要全盘格式化一遍,不然安装过程中会出现无法分区,显示该盘的空间也会异常
安装CUDA toolkit 9.0和cuDNN 7.1
参考:
https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html
https://docs.nvidia.com/cuda/archive/9.0/cuda-installation-guide-linux/index.html(推荐)
[外链图片转存失败(img-fsWLqjEp-1568858900082)(media/939af5add09a4538dfdd59f688ba6f75.png)]
安装CUDA Toolkit 9.0
- 通过下面命令产看机器的GPU硬件型号,然后在http://developer.nvidia.com/cuda-gpus这里查看机器的GPU是否支持CUDA。
$ lspci | grep -i nvidia
- 通过下面命令查看gcc版本,确保版本是4.xx
$ gcc --version
- 如果当前gcc是5.xx最好将版本降至4.xx,命令如下:
- 下载gcc/g++ 4.9.x
$ sudo apt-get install -y gcc-4.9
$ sudo apt-get install -y g+±4.9
- 链接gcc/g++实现降级
$ cd /usr/bin
$ sudo rm gcc
$ sudo ln -s gcc-4.9 gcc
$ sudo rm g++
$ sudo ln -s g+±4.9 g++
- CUDA Kit 9.0下载地址
https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&target_arch=x86_64&target_distro=Ubuntu&target_version=1604
[外链图片转存失败(img-ZnkMhVrH-1568858900083)(media/3f1de890ef2015bec7ba8bc8b9eb5425.png)]
下载deb后,通过下面几个步骤安装:
[外链图片转存失败(img-BA0l6W6u-1568858900085)(media/94013053f335a7bd82022782bfa5b578.png)]
然后将/usr/local/cuda-9.0/bin加入PATH环境变量。对应Ubuntu,该路径是加入/etc/profile:
[外链图片转存失败(img-swWVImVE-1568858900086)(media/c259a7edaad218520d62099e73f57463.png)]
[外链图片转存失败(img-pOQXOnsN-1568858900086)(media/72097857a95532dddb4756e155fb9e47.png)]
还需要安装cuda-command-line-tools,安装完之后加入/etc/profile中。
启动进程守护,且本地生成一套CUDA的samples程序:
[外链图片转存失败(img-dzlNdThE-1568858900088)(media/b1515bac45c65f52e062c57f17475f66.png)]
注意:
安装完CUDA
Toolkit后需要重启账户ssh连接,cuda-install-samples-9.1.sh才能够识别上图的例子是9.1,但是真正应该是9.0
到这里,测试nvcc应该可以证明有效。
安装cuDNN 7.1
如果需要手动编译Tensorflow,就需要安装cuDNN,对应的版本选择了7.1,下载地址:
https://developer.nvidia.com/rdp/cudnn-download
这里下载需要在官网上注册一个帐号(注册不一定那么顺利),然后从中选择正确版本安装:
[外链图片转存失败(img-NEMpsdy4-1568858900088)(media/bff2ff5e5504f4e49782bd358ea57d88.png)]
解压压缩包:
tar -xvf cudnn-9.0-linux-x64-v7.1.tgz
然后将头文件和库文件拷贝至相关路径:
sudo cp cuda/include/cudnn.h /usr/local/cuda-9.0/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda-9.0/lib64/
sudo chmod a+r /usr/local/cuda-9.0/include/cudnn.h
sudo chmod a+r /usr/local/cuda-9.0/lib64/libcudnn*
安装Anaconda 3 x64
为什么要安装Anaconda?因为可以控制多个版本的Python。
下载地址:https://www.anaconda.com/download/#linux
[外链图片转存失败(img-MQ6IvUK9-1568858900089)(media/5cdea2ce3a1e677f5951e77d3d1bc2e7.png)]
Anaconda通常安装在Ubuntu的公共目录:/usr/local/data/anaconda3/,然后将/usr/local/data/anaconda3/bin/加入/etc/profile中:
[外链图片转存失败(img-dCHd2VTv-1568858900090)(media/5a4426c8e5c44cb2969da75e71af5edf.png)]
# cuda export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-9.0/extras/CUPTI/lib64 # anaconda PATH=/usr/local/data/anaconda3/bin:$PATH # bazel PATH=/usr/local/data/bazel-0.13.0/lib/bazel/bin:$PATH export PATH |
---|
- 为了提高cuda安装包下载速度,增加Anaconda在中国的镜像:
[外链图片转存失败(img-zt1tHuLJ-1568858900091)(media/c43e37461448dd3fcf6c6f04660f7a24.png)]
sudo conda config --add channels
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
sudo conda config --add channels
https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
sudo conda config --set show_channel_urls yes
安装Google的构建工具Bazel
参考:https://docs.bazel.build/versions/master/install-ubuntu.html#install-with-installer-ubuntu
[外链图片转存失败(img-fyJxVRxO-1568858900092)(media/3bacbed471f71301ecb1122eca4d8706.png)]
说明:
Step 1:其中的python是安装在Anacond3中的3.6
Step 2-3:实际直接执行sudo bash
bazel-<version>-installer-linux-x86_64.sh即可,安装位置通过–prefix=/usr/local/data/bazel-0.13.0
Step 4:将路径加入/etc/profile
[外链图片转存失败(img-haShRCIb-1568858900093)(media/5b6e2ae6ae1317f1eabc6b95768dd269.png)]
编译Tensorflow 17.1
参考:https://www.tensorflow.org/install/install_sources#ConfigureInstallation
为安装进行配置
$ cd tensorflow # cd to the top-level directory created
$ ./configure
Please specify the location of python. [Default is /usr/bin/python]: /usr/bin/python2.7 Found possible Python library paths: /usr/local/lib/python2.7/dist-packages /usr/lib/python2.7/dist-packages Please input the desired Python library path to use. Default is [/usr/lib/python2.7/dist-packages] Using python library path: /usr/local/lib/python2.7/dist-packages Please specify optimization flags to use during compilation when bazel option “–config=opt” is specified [Default is -march=native]: Do you wish to use jemalloc as the malloc implementation? [Y/n] jemalloc enabled Do you wish to build TensorFlow with Google Cloud Platform support? [y/N] No Google Cloud Platform support will be enabled for TensorFlow Do you wish to build TensorFlow with Hadoop File System support? [y/N] No Hadoop File System support will be enabled for TensorFlow Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N] No XLA support will be enabled for TensorFlow Do you wish to build TensorFlow with VERBS support? [y/N] No VERBS support will be enabled for TensorFlow Do you wish to build TensorFlow with OpenCL support? [y/N] No OpenCL support will be enabled for TensorFlow Do you wish to build TensorFlow with CUDA support? [y/N] Y CUDA support will be enabled for TensorFlow Do you want to use clang as CUDA compiler? [y/N] nvcc will be used as CUDA compiler Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.0 Please specify the location where CUDA 9.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7 Please specify the location where cuDNN 7 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]: Please specify a list of comma-separated Cuda compute capabilities you want to build with. You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Please note that each additional compute capability significantly increases your build time and binary size. [Default is: “3.5,5.2”]: 3.0 Do you wish to build TensorFlow with MPI support? [y/N] MPI support will not be enabled for TensorFlow Configuration finished |
---|
构建和安装Tensorflow的pip包(whl)
http://cwiki.apachecn.org/pages/viewpage.action?pageId=10029599
[外链图片转存失败(img-oKU2joVH-1568858900093)(media/0edbc4699c42688922c7fd478958d5f7.png)]
【注意】
bazel
build --config=opt //tensorflow/tools/pip_package:build_pip_package --verbose-failures
编译是一个漫长的过程,中间没有报错就表示编译成功。
[外链图片转存失败(img-FkETH2mm-1568858900094)(media/b2dcb6a2286c1b05a38f9f599cfd7a01.png)]
【注意】
安装时,最好使用root账户
同时,有可能需要更新pip,将其更新至最新版本,更新后查看pip版本,注意是否是更新至anaconda3的python目录
测试安装的Tensorflow
[外链图片转存失败(img-A0kolKp9-1568858900095)(media/943db0e75bf1d2c708f6fae264cf2ad5.png)]
安装Keras
sudo su root 切换至root账户
pip install kereas
附:从源代码安装 TensorFlow【官网】
[外链图片转存失败(img-tZXkrDLD-1568858900096)(media/34a2982c366d4a356d1e4d5b9f503029.png)]
[外链图片转存失败(img-8c4Q29rF-1568858900097)(media/42da60daed3058515e0e05977b6eae1f.png)]
CentOS 6.9安装Tensorflow
yum安装gcc 4.9.2 (devtoolset)
[外链图片转存失败(img-cAfDIYXk-1568858900098)(media/694750f6e09d165939663cc35c8154e0.png)]
https://blog.csdn.net/ysx_cpp/article/details/77187453
【注意】上图中最后一句话:将其添加至环境变量中
安装CUDA Toolkit9.0
[外链图片转存失败(img-RrQlrAO6-1568858900099)(media/29af4a7b31f13b87b877366312c610cb.png)]
https://developer.nvidia.com/cuda-90-download-archive?target_os=Linux&target_arch=x86_64&target_distro=CentOS&target_version=6&target_type=rpmlocal
sudo rpm -i cuda-repo-rhel6-9-0-local-9.0.176-1.x86_64.rpm sudo yum clean all sudo yum install cuda |
---|
Install kernel-headers rpm安装包
- 直接安装
$ sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
- 下载rpm安装
kernel-headers下载地址:https://rpmfind.net/linux/rpm2html/search.php?query=kernel-headers
$ sudo rpm -ivh kernel-headers-2.6.32-696.16.1.el6.x86_64.rpm
测试安装nvidia驱动
/usr/local/cuda-9.1/extras/demo_suite/deviveQuery
nvidia-smi
编译cuda例子,执行查看cuda参数
安装好之后可能需要重启机器
安装Anaconda3
默认自动添加至/root/.bashrc
[外链图片转存失败(img-s4mDx3Jx-1568858900100)(media/98486a513a4cc2d894124b1a418fd017.png)]
为了保证其他用可用,通常将其配置在/etc/profile中
# gcc 4.9 devtoolset source /opt/rh/devtoolset-3/enable # cuda export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} # anaconda export PATH=/usr/local/data/anaconda3/bin:$PATH # bazel export PATH=/usr/local/data/bazel-0.13.0/lib/bazel/bin:$PATH # JAVA export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el6_9.x86_64/ export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH |
---|
源码安装bazel
https://docs.bazel.build/versions/master/install-compile-source.html
[外链图片转存失败(img-Uf6jaYzT-1568858900101)(media/c74a714ab70cfcbc6fc14e054340ab88.png)]
安装java
卸载原来java
[root@192 ~]# yum list installed |grep java
[root@192 ~]# yum -y remove java-1.8.0-openjdk*
*表时卸载所有openjdk相关文件输入
[root@192 ~]# yum -y remove tzdata-java.noarch 卸载tzdata-java
安装新java
[root@192 ~]# yum -y list java*
[root@192 ~]# yum install java-1.8.0-openjdk java-1.8.0-openjdk-devel 安装JDK,如果没有java-1.8.0-openjdk-devel就没有javac命令
[root@192 ~]# java -version 查看Java版本信息
添加java环境
export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.171-8.b10.el6_9.x86_64/
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH
添加至/etc/profile
查看glibc的版本
strings /lib64/libc.so.6 | grep GLIBC
ldd --version
GPU1 Glibc 2.17 没有发现错误
Tensorflow编译错误
错误1:patch
- build tensorflow源码出现patch command not found,则需要安装sudo yum install
patch
错误2:ld链接错误
- crosstool_wrapper_driver_is_not_gcc failed
/usr/bin/ld: unrecognized option ‘-plugin’
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
cd
/home/s915_chenhao/.cache/bazel/_bazel_s915_chenhao/95f58aebcb4f1dd76c69c9695ad4bca2/external/local_config_cuda/crosstool
cd
/home/s915_chenhao/.cache/bazel/_bazel_s915_chenhao/95f58aebcb4f1dd76c69c9695ad4bca2/execroot/org_tensorflow/
external/local_config_cuda/crosstool/clang/bin
[外链图片转存失败(img-st1MiVl8-1568858900101)(media/2d5bf0aff54197764e73e9c9c7959c5c.png)]
[外链图片转存失败(img-EhNyNGh8-1568858900102)(media/48d00ea442b287ce93dad64f9b7d1086.png)]
错误3:找不到库libcudnn.so
[外链图片转存失败(img-mmzJe7Ft-1568858900104)(media/5c468032a0197e2d9638ed75cfa76f1c.png)]
附录
NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver …
- 我的记录
- 安装所有kernel,sudo yum install kernel*
或者sudo yum install kernel-devel-$(uname -r) kernel-headers-$(uname -r)
- 卸载重装cuda kit,sudo yum remove cuda*
然后Install repository meta-data
$ sudo rpm --install cuda-repo-<distro>-<version>.<architecture>.rpm
Clean Yum repository cache
$ sudo yum clean expire-cache
Install CUDA
$ sudo yum install cuda
重启机器,sudo reboot
编译sample
cuda-install-samples-9.0.sh ./
make
[deviceQueryDrv]$ ./deviceQueryDrv
- 参考https://zhuanlan.zhihu.com/p/32958364
[外链图片转存失败(img-AFolU1XM-1568858900104)(media/5aaa66288eca50bbb74a08eceaa48f67.jpg)]
CentOS安装PyTorch
假设已经安装好Nvidia驱动和cuDNN
安装PyTorch
直接使用conda
install可能会出现网络问题,因此可以首先下载PyTorch安装包,然后离线安装。
下载安装包
使用IDM多线程下载速度还挺快,下载地址:
https://conda.anaconda.org/pytorch/linux-64/pytorch-0.4.1-py36_cuda9.0.176_cudnn7.1.2_1.tar.bz2
conda离线安装
- 下载地址:https://anaconda.org/pytorch/pytorch/files
[外链图片转存失败(img-Uy2OXaS5-1568858900105)(media/b263beb788d680313c5f57f61e06cfd5.png)]
conda install --offline pytorch-0.4.1-py36_cuda9.0.176_cudnn7.1.2_1.tar.bz2
测试安装
import torch
x = torch.Tensor(5, 3)
x = torch.rand(5, 3)
x
没有报错表示安装成功。
安装torchvision
https://zllrunning.github.io/2018/04/17/20180417/
pip install visdom
python -m visdom.server
TensorFlow源码学习
[外链图片转存失败(img-t7bK2BNf-1568858900106)(media/ce278fe12c47805efb8a7703a3576469.png)]
pywrap_tensorflow_internal.cc是swig生成的c++代码,pywrap_tensorflow_internal.py是生成的py代码,用于调用pywrap_tensorflow_internal.so模块。
参考
- Tensorflow[源码安装时bazel行为解析]
[外链图片转存失败(img-m4OoCpX2-1568858900106)(media/e070be08072ec78fa0d67353402c46e3.png)]
- 带你深入AI(6)-
详解bazel
[外链图片转存失败(img-gEIzqsYn-1568858900107)(media/59dacb10b9df953762271256b1d086db.png)]
Ubuntu和Centos安装Tensorflow教程PyTorch相关推荐
- linux安装tensorflow教程,Ubuntu 16.04 安装 TensorFlow(GPU支持)
本文记录Ubuntu 16.04安装Tensorflow步骤,也包括怎么从源码编译安装Tensorflow. 要想安装Tensorflow GPU版本,你需要有一个新一点的Nvidia显卡. Tens ...
- 人工智能学习(二)安装tensorflow与pytorch
安装tensorflow与pytorch 前提: 已安装完Anaconda 这⾥主要使⽤ Windows 中的Anaconda Prompt 进⾏安装,因为在Linux.Mac通⽤也是⼤同⼩异的 第一 ...
- Ubuntu 20.04 安装Tensorflow GPU版本 (GTX-1060显卡)
Ubuntu 20.04 安装Tensorflow GPU版本 (GTX-1060显卡) 参考: https://cyfeng.science/2020/05/02/ubuntu-install-nv ...
- centos安装tensorFlow的java环境
参考问题汇总 centos安装tensorFlow版本的时候会遇到的一些问题,参考这个链接:https://blog.csdn.net/luoyexuge/article/details/783212 ...
- CentOS安装Redis教程
CentOS安装Redis教程 从官网下载Redis:https://redis.io/download 进入下载文件夹,解压安装包到opt目录下:tar -zxvf redis-4.0.2.tar. ...
- Win10下Anaconda3上安装tensorflow教程
Win10下Anaconda3上安装tensorflow教程 大概花费了4-5个小时的时间安装上了基于GPU版本的tensorflow1.3.0(除此之外还有基于CPU的版本),大体安装步骤如下: s ...
- Ubuntu 18.04安装UHD教程
Ubuntu 18.04安装UHD教程 准备阶段 首先将Ubuntu的下载源给替换了,默认的下载源是国外的,如果不替换更新下载速度特别慢,并且会导致后边的步骤出现各种错误,所以这一步很重要.具体操作是 ...
- ubuntu 16.04 安装TensorFlow GPU版本
推荐新版安装教程 http://blog.csdn.net/chenhaifeng2016/article/details/78874883 在ubuntu 16.04上安装cuda8.0和cudnn ...
- Ubuntu 15.04 安装TensorFlow(源码编译) 及测试梵高作画
介绍Google的TensorFlow机器学习开源库,在UbuntuKylin上的安装和和源码编译. 原始官方文档参见:http://www.tensorflow.org. 本电脑配置如下: 3.19 ...
最新文章
- Go 学习笔记(47)— Go 标准库之 strconv(string/int 互相转换、Parse 字符串转换为指定类型、Format 指定类型格式化为字符串)
- 背包模型dp1:01背包,完全背包,多重背包的两大优化的详解
- git克隆远程项目并创建本地对应分支
- 神马搜索上线奥运会人气榜单 中国女排包揽前三
- 格兰因果模型可以分析哪些东西_如何系统地学习统计学,指导入门数据分析
- 万网控制面板 php_UPUPW 控制面板启动关闭区介绍
- 算法 --- 平衡二叉树
- [CF903G]Yet Another Maxflow Problem
- 信息学奥数一本通(1170:计算2的N次方)
- Ubuntu16 python2.7升级python3.5
- 验毛坯房要注意什么?
- windows 代理软件_MacOS好用软件推荐(一)
- android 入门-git之上传本地代码到github
- FFmpeg教程(超级详细版)
- 情侣推送早安问候之小程序订阅通知模板推送 (出现在消息列表的服务通知)
- 计算机专业学生组装电脑配置,我打算配置一台5000左右的组装台式电脑
- 引导从打游戏到学计算机,【润笙学堂】暑假孩子沉迷游戏?妈妈一招化解!
- 第三方支付机构有哪些?他们的资金是如何运作保障用户的资金安全的?
- 同构网络vs异构网络
- 目录——Java趣味编程100例