hadoop 源码编译---Win7(64位)
2024-04-13 08:25:29
说明：
在Windows 平台上编译hadoop 源代码，hadoop源码包的安装说明文件【BUILDING.txt】描述如下
haoop官方文档上需要准备的文件:
* Windows System
* JDK 1.7+
* Maven 3.0 or later
* Findbugs 1.3.9 (if running findbugs)
* ProtocolBuffer 2.5.0
* CMake 2.6 or newer
* Windows SDK 7.1 or Visual Studio 2010 Professional【没用到，我用cmd代替】
* Windows SDK 8.1 (if building CPU rate control for the container executor)【没用到，windows sdk7 和windows sdk8 二选一】
* zlib headers (if building native code bindings for zlib)【没用到】
* Internet connection for first build (to fetch all Maven and Hadoop dependencies)【我用的是nexus-2.14.1-01-bundle.zip】
* Unix command-line tools from GnuWin32: sh, mkdir, rm, cp, tar, gzip.【没用到】
These
tools must be present on your PATH.
1.我的准备文件：
(1).apache-maven-3.0.4
(2).protoc-2.5.0-win32.zip
(3).findbugs-1.3.9.zip
(4).cmake-3.8.0-rc2-win64-x64.zip
2.系统变量Path设置
F:\Maven\apache-maven-3.0.4-bin\apache-maven-3.0.4\bin;
E:\Linux\google-protobuf-2.5.0\protoc-2.5.0-win32;
E:\Linux\findbugs-1.3.9\bin;
E:\Linux\cmake-3.8.0-rc2-win64-x64\bin;
3. 首先进入hadoop源码文件夹--->hadoop-maven-plugins
【我的：E:\Linux\hadoop-2.7.3-src\hadoop-maven-plugins】 在地址栏输入cmd 后回车，进入当前目录
4. 运行mvn clean install 下载编译包
5.退会到hadoop源码根目录执行：
mvn eclipse:eclipse -DskipTests
成功后的界面输入出如下：
main:[mkdir] Created dir: E:\Linux\hadoop-2.7.3-src\hadoop-tools\hadoop-tools-dist\target\test-dir[mkdir] Created dir: E:\Linux\hadoop-2.7.3-src\hadoop-tools\hadoop-tools-dist\target\test\data [INFO] Executed tasks [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-tools-dist --- [INFO] [INFO] <<< maven-eclipse-plugin:2.10:eclipse (default-cli) < generate-resources @ hadoop-tools-dist <<< [INFO] [INFO] --- maven-eclipse-plugin:2.10:eclipse (default-cli) @ hadoop-tools-dist --- [INFO] Using Eclipse Workspace: null [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.7 [INFO] Wrote settings to E:\Linux\hadoop-2.7.3-src\hadoop-tools\hadoop-tools-dist\.settings\org.eclipse.jdt.core.prefs [INFO] File E:\Linux\hadoop-2.7.3-src\hadoop-tools\hadoop-tools-dist\.project already exists.Additional settings will be preserved, run mvn eclipse:clean if you want old settings to be removed. [INFO] Wrote Eclipse project for "hadoop-tools-dist" to E:\Linux\hadoop-2.7.3-src\hadoop-tools\hadoop-tools-dist. [INFO] [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop Tools 2.7.3 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] >>> maven-eclipse-plugin:2.10:eclipse (default-cli) > generate-resources @ hadoop-tools >>> [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-tools --- [INFO] Executing tasksmain:[mkdir] Created dir: E:\Linux\hadoop-2.7.3-src\hadoop-tools\target\test-dir [INFO] Executed tasks [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-tools --- [INFO] [INFO] <<< maven-eclipse-plugin:2.10:eclipse (default-cli) < generate-resources @ hadoop-tools <<< [INFO] [INFO] --- maven-eclipse-plugin:2.10:eclipse (default-cli) @ hadoop-tools --- [INFO] Not running eclipse plugin goal for pom project [INFO] Using Eclipse Workspace: null [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.7 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Apache Hadoop Distribution 2.7.3 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] >>> maven-eclipse-plugin:2.10:eclipse (default-cli) > generate-resources @ hadoop-dist >>> [INFO] [INFO] --- maven-antrun-plugin:1.7:run (create-testdirs) @ hadoop-dist --- [INFO] Executing tasksmain:[mkdir] Created dir: E:\Linux\hadoop-2.7.3-src\hadoop-dist\target\test-dir [INFO] Executed tasks [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hadoop-dist --- [INFO] [INFO] <<< maven-eclipse-plugin:2.10:eclipse (default-cli) < generate-resources @ hadoop-dist <<< [INFO] [INFO] --- maven-eclipse-plugin:2.10:eclipse (default-cli) @ hadoop-dist --- [INFO] Using Eclipse Workspace: null [INFO] Adding default classpath container: org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.7 [INFO] Wrote settings to E:\Linux\hadoop-2.7.3-src\hadoop-dist\.settings\org.eclipse.jdt.core.prefs [INFO] File E:\Linux\hadoop-2.7.3-src\hadoop-dist\.project already exists.Additional settings will be preserved, run mvn eclipse:clean if you want old settings to be removed. [INFO] Wrote Eclipse project for "hadoop-dist" to E:\Linux\hadoop-2.7.3-src\hadoop-dist. [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Hadoop Main ................................. SUCCESS [ 0.424 s] [INFO] Apache Hadoop Build Tools .......................... SUCCESS [ 0.215 s] [INFO] Apache Hadoop Project POM .......................... SUCCESS [ 0.520 s] [INFO] Apache Hadoop Annotations .......................... SUCCESS [ 0.108 s] [INFO] Apache Hadoop Project Dist POM ..................... SUCCESS [ 0.084 s] [INFO] Apache Hadoop Assemblies ........................... SUCCESS [ 0.095 s] [INFO] Apache Hadoop Maven Plugins ........................ SUCCESS [ 1.108 s] [INFO] Apache Hadoop MiniKDC .............................. SUCCESS [ 3.725 s] [INFO] Apache Hadoop Auth ................................. SUCCESS [ 3.602 s] [INFO] Apache Hadoop Auth Examples ........................ SUCCESS [ 0.471 s] [INFO] Apache Hadoop Common ............................... SUCCESS [ 10.259 s] [INFO] Apache Hadoop NFS .................................. SUCCESS [ 13.015 s] [INFO] Apache Hadoop KMS .................................. SUCCESS [ 10.464 s] [INFO] Apache Hadoop Common Project ....................... SUCCESS [ 0.070 s] [INFO] Apache Hadoop HDFS ................................. SUCCESS [ 5.865 s] [INFO] Apache Hadoop HttpFS ............................... SUCCESS [ 32.714 s] [INFO] Apache Hadoop HDFS BookKeeper Journal .............. SUCCESS [ 5.733 s] [INFO] Apache Hadoop HDFS-NFS ............................. SUCCESS [ 1.789 s] [INFO] Apache Hadoop HDFS Project ......................... SUCCESS [ 0.088 s] [INFO] hadoop-yarn ........................................ SUCCESS [ 0.066 s] [INFO] hadoop-yarn-api .................................... SUCCESS [ 0.710 s] [INFO] hadoop-yarn-common ................................. SUCCESS [ 8.698 s] [INFO] hadoop-yarn-server ................................. SUCCESS [ 0.065 s] [INFO] hadoop-yarn-server-common .......................... SUCCESS [ 5.843 s] [INFO] hadoop-yarn-server-nodemanager ..................... SUCCESS [ 2.519 s] [INFO] hadoop-yarn-server-web-proxy ....................... SUCCESS [ 0.636 s] [INFO] hadoop-yarn-server-applicationhistoryservice ....... SUCCESS [ 0.958 s] [INFO] hadoop-yarn-server-resourcemanager ................. SUCCESS [ 3.172 s] [INFO] hadoop-yarn-server-tests ........................... SUCCESS [ 13.857 s] [INFO] hadoop-yarn-client ................................. SUCCESS [ 2.377 s] [INFO] hadoop-yarn-server-sharedcachemanager .............. SUCCESS [ 2.371 s] [INFO] hadoop-yarn-applications ........................... SUCCESS [ 0.070 s] [INFO] hadoop-yarn-applications-distributedshell .......... SUCCESS [ 0.777 s] [INFO] hadoop-yarn-applications-unmanaged-am-launcher ..... SUCCESS [ 0.478 s] [INFO] hadoop-yarn-site ................................... SUCCESS [ 0.069 s] [INFO] hadoop-yarn-registry ............................... SUCCESS [ 1.384 s] [INFO] hadoop-yarn-project ................................ SUCCESS [ 0.477 s] [INFO] hadoop-mapreduce-client ............................ SUCCESS [ 0.227 s] [INFO] hadoop-mapreduce-client-core ....................... SUCCESS [ 5.169 s] [INFO] hadoop-mapreduce-client-common ..................... SUCCESS [ 7.680 s] [INFO] hadoop-mapreduce-client-shuffle .................... SUCCESS [ 3.952 s] [INFO] hadoop-mapreduce-client-app ........................ SUCCESS [ 1.673 s] [INFO] hadoop-mapreduce-client-hs ......................... SUCCESS [ 5.814 s] [INFO] hadoop-mapreduce-client-jobclient .................. SUCCESS [ 1.755 s] [INFO] hadoop-mapreduce-client-hs-plugins ................. SUCCESS [ 0.515 s] [INFO] Apache Hadoop MapReduce Examples ................... SUCCESS [ 4.946 s] [INFO] hadoop-mapreduce ................................... SUCCESS [ 0.253 s] [INFO] Apache Hadoop MapReduce Streaming .................. SUCCESS [ 1.095 s] [INFO] Apache Hadoop Distributed Copy ..................... SUCCESS [05:56 min] [INFO] Apache Hadoop Archives ............................. SUCCESS [ 3.804 s] [INFO] Apache Hadoop Rumen ................................ SUCCESS [ 0.891 s] [INFO] Apache Hadoop Gridmix .............................. SUCCESS [ 7.510 s] [INFO] Apache Hadoop Data Join ............................ SUCCESS [ 0.351 s] [INFO] Apache Hadoop Ant Tasks ............................ SUCCESS [ 0.483 s] [INFO] Apache Hadoop Extras ............................... SUCCESS [ 0.567 s] [INFO] Apache Hadoop Pipes ................................ SUCCESS [ 0.067 s] [INFO] Apache Hadoop OpenStack support .................... SUCCESS [ 1.997 s] [INFO] Apache Hadoop Amazon Web Services support .......... SUCCESS [01:30 min] [INFO] Apache Hadoop Azure support ........................ SUCCESS [ 5.079 s] [INFO] Apache Hadoop Client ............................... SUCCESS [ 3.223 s] [INFO] Apache Hadoop Mini-Cluster ......................... SUCCESS [ 1.524 s] [INFO] Apache Hadoop Scheduler Load Simulator ............. SUCCESS [ 7.012 s] [INFO] Apache Hadoop Tools Dist ........................... SUCCESS [ 7.261 s] [INFO] Apache Hadoop Tools ................................ SUCCESS [ 0.067 s] [INFO] Apache Hadoop Distribution ......................... SUCCESS [ 0.272 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 10:56 min [INFO] Finished at: 2017-03-20T08:39:17+08:00 [INFO] Final Memory: 77M/247M [INFO] ------------------------------------------------------------------------
编译输出的目录：
转载于:https://www.cnblogs.com/zhangxiaolin/p/6585982.html
hadoop 源码编译---Win7(64位)相关推荐
- 大数据之-Hadoop源码编译_源码编译的意义---大数据之hadoop工作笔记0044
然后我们再来看一下hadoop的源码编译, 先看看,可以去官网去下载,hadoop-2.7.2.tar.gz的源码去 下载了以后我们需要去编译,源码,为什么? 比如:上面这个源码是32位的,那么如果我 ...
- hadoop编译java,Hadoop源码编译(2.4.1)
背景 Hadoop 2.4.1预编译版本自带的libhadoop.so是在32位机器上编译上,导致在64位OS上运行时,总出现如下告警: WARN util.NativeCodeLoader: Una ...
- Hadoop源码编译介绍-尚硅谷大数据培训
前期准备工作 1.CentOS联网 使用能连接外网的CentOS系统虚拟机,使用root用户进行编译,避免出现文件夹权限问题. 2.Jar包准备 从官网下载以下安装包 (1)hadoop-3.1.3- ...
- Hadoop-2.8.0集群搭建、hadoop源码编译和安装、host配置、ssh免密登录、hadoop配置文件中的参数配置参数总结、hadoop集群测试,安装过程中的常见错误
25. 集群搭建 25.1 HADOOP集群搭建 25.1.1集群简介 HADOOP集群具体来说包含两个集群:HDFS集群和YARN集群,两者逻辑上分离,但物理上常在一起 HDFS集群: 负责海量数据 ...
- windows下使用MingW-w64 编译 Qt4.8.6源码，生成64位QT库
1. 安装mingw-w64 具体安装教程,可以参考:https://blog.csdn.net/wo198711203217/article/details/105032057 安装时记得选择x86 ...
- 大数据之-Hadoop源码编译_源码编译具体流程_以及编译步骤---大数据之hadoop工作笔记0046
然后我们开始来编译hadoop的源码,首先我们 先做一个干净的虚拟机,我们克隆准备好的CentOS-0710-100就可以了 选择下一步 选择创建完整克隆,点击下一步
- 大数据之-Hadoop源码编译_编译hadoop源码前_需要准备的软件_以及编译步骤---大数据之hadoop工作笔记0045
编译源码,首先要准备一台Centos可以联网的虚拟机,一定要可以连接外网 然后准备上面的这些软件. 可以看到软件都在上面这个目录下. 首先是安装jdk,然后配置JAVA_HOME
- 使用源码编译64位的3D Slicer（综述）
3D Slicer是一款开源的用于医学图像分析(主要是分割和配准)与可视化成像的软件.我们可以把自己插件加入到3D Slicer软件中来实现我们想要的功能,它向我们提供一个检测算法与改进算法的平台.官 ...
- win7源码编译cef3.2743
1.前言 最近项目中要用到一个内嵌浏览器,想要尝试下使用源码进行编译,历经一个晚上和一个上午的时间终于搞定,这里记录下过程 2.准备 2.1 win7 64位 旗舰版 2.2 vs2015 updat ...
最新文章
- 卷积神经网络（cnn） 手写数字识别
- linux上面的sz,rz命令与ssh的配合
- 35+ Top Apache Tomcat Interview Questions And Answers【转】
- vue 引入swiper4,在有些手机Dom7会有问题
- 八十八、Webpack打包工具
- 观察者模式在个人网站邮件发布与订阅中的实践
- 宜搭小技巧｜自动计算日期时长，3个公式帮你敲定
- KafkaController介绍
- 蓝桥杯 ALGO-23 算法训练 一元三次方程求解
- php 返回josn 中文,php返回json数据中文显示不友好的问题的解决办法
- 462.最少移动次数使数组元素相等II
- 分享win10常用图标ico_png_html素材
- 如何把很多照片拼成一张照片_如何将多张图片合成一个PDF文件
- DongDong认亲戚（map+并查集）
- ELI'S CURIOUS MIND
- 单目标跟踪CVPR 2018 ECO+
- 计算机控制台如何打开,Windows控制台是什么,如何打开Win7控制台?
- Android 实时监听耳机拔除
- A beginning Flags
- [转]Windows 下常用盗版软件的替代免费软件列表
热门文章
- python c 混合编程 用c循环_混合编程：用 C 语言来扩展 Python 大法吧！
- centos MySQL 双机_CentOS利用Keepalived构建双主MySQL+双机热备
- 【cmd】日期、时间格式化
- Linux mkdir 命令创建多级目录
- 【thymeleaf】Access to request parameters is forbidden in this context
- android ontoch事件无反应_一切从android的handler说起(三)
- android 仿微信选取相册_Android 仿微信 相册多图选择器
- 软件开发生命周期中的设计阶段_测试基础——软件开发生命周期
- python文本去重_Python做文本按行去重的实现方法
- java 生成 xml dtd_使用DTD文件中的JAXB生成Java类 – 如何修改DTD？