一、集群软硬件环境准备：

操作系统：  centos 7 x86_64.1804

Ambari版本：2.7.0

HDP版本：3.0.0

HAWQ版本：2.3.0
5台PC作为工作站：

ep-bd01 ep-bd02 ep-bd03 ep-bd04 ep-bd05

其中ep-bd01作为主节点，用于安装ambari-server。

二、配置操作系统，安装必备软件

1，安装CentOS 7操作系统：环境配置，安装必备软件。

2，安装配置NTP服务，保证集群时间保持同步，以防止由于时间不同而造成掉线故障。

详细看随笔：基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库之——安装配置NTP服务，保证集群时间保持同步
见《安装配置NTP服务》

3，安装MariaDB Server用于Ambari server以及Hue和Hive

详细过程，参见：基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库之一 —— MariaDB 安装配置

4，安装yum priorities plugin

yum install yum-plugin-priorities -y

三、搭建本地仓库：

1，下载软件包：

cd /rootmkdirdownloads
cd downloadswget http://public-repo-1.hortonworks.com/HDP-GPL/centos7/3.x/updates/3.0.0.0/HDP-GPL-3.0.0.0-centos7-gpl.tar.gz
wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/hdp.repo
wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/HDP-3.0.0.0-1634.xml
wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.0.0/ambari.repo
wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.0.0/HDP-3.0.0.0-centos7-rpm.tar.gz
wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.0.0/ambari-2.7.0.0-centos7.tar.gz

2、搭建本地仓库：

安装并开启Apache HTTP服务

yum install httpd -y
systemctl enable httpd
systemctl start httpd

安装HTTPD服务

确保/var/www/html目录存在，没有的话创建。

mkdir -p /var/www/html

创建HDP，HDF子目录

cd /var/www/htmlmkdir hdp  hdf

解开下载的软件包：

cd /var/www/htmltar -zxvf  /root/downloads/ambari-2.7.0.0-centos7.tar.gz    -C .tar -zxvf  /root/downloads/HDP-3.0.0.0-centos7-rpm.tar.gz   -C ./hdptar -zxvf  /root/downloads/HDP-GPL-3.0.0.0-centos7-gpl.tar.gz  -C ./hdptar -zxvf  /root/downloads/HDP-UTILS-1.1.0.22-centos7.tar.gz   -C ./hdp

修改下载的ambari.repo，

vim ambari.repo

安装如下内容修改，[注意版本号，需要根据具体下载的版本不同修改，解压后自己查看一下]：
#VERSION_NUMBER=2.7.0.0-897

[ambari-2.7.0.0]
#json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
name=ambari Version - ambari-2.7.0.0
baseurl=http://ep-bd01/ambari/centos7/2.7.0.0-897
gpgcheck=1
gpgkey=http://ep-bd01/ambari/centos7/2.7.0.0-897/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1



复制到/etc/yum.repos.d




cp ambari.repo   /etc/yum.repos.d/ambari.repo


修改下载的hdp.repo，




vim hdp.repo


安装如下内容修改，[注意版本号，需要根据具体下载的版本不同修改，解压后自己查看一下]：




#VERSION_NUMBER=3.0.0.0-1634

            [HDP-3.0]

            name=HDP Version - HDP-3.0.0.0

            baseurl=http://ep-bd01/hdp/HDP/centos7/3.0.0.0-1634

            gpgcheck=1

            gpgkey=http://ep-bd01/hdp/HDP/centos7/3.0.0.0-1634/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

            enabled=1

            priority=1


[HDP-3.0-GPL]

            name=HDP GPL Version - HDP-GPL-3.0.0.0

            baseurl=http://ep-bd01/hdp/HDP-GPL/centos7/3.0.0.0-1634

            gpgcheck=1

            gpgkey=http://ep-bd01/hdp/HDP-GPL/centos7/3.0.0.0-1634/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

            enabled=1

            priority=1


[HDP-UTILS-1.1.0.22]

            name=HDP-UTILS Version - HDP-UTILS-1.1.0.22

            baseurl=http://ep-bd01/hdp/HDP-UTILS/centos7/1.1.0.22

            gpgcheck=1

            gpgkey=http://ep-bd01/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins

            enabled=1

            priority=1


保存退出，复制到/etc/yum.repos.d/




cp  hdp.repo  /etc/yum.repos.d/hdp.repo


四、主节点安装ambari server


1，使用刚才配置好的本地仓库，直接yum命令安装。




yum install ambari-server -y


2，查看ambari server 状态




 systemctl status ambari-server ● ambari-server.service - LSB: ambari-server daemonLoaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled)Active: inactive (dead)Docs:man:systemd-sysv-generator(8)


看到ambari server已成功安装了。


3，配置mariadb，建立用户和数据库供ambari使用


建立数据库用户ambari




mysql -uroot -p


MariaDB [(none)]> use mysql;

            Reading table information for completion of table and column names

            You can turn off this feature to get a quicker startup with -A


Database changed

            MariaDB [mysql]> grant all privileges on *.* to 'ambari'@'%' identified by 'ambari';

            Query OK, 0 rows affected (0.06 sec)


MariaDB [mysql]> flush privileges;

            Query OK, 0 rows affected (0.00 sec)


MariaDB [mysql]>


建立数据库ambari，并运行ambari数据库建表sql命令文件。




MariaDB [mysql]>create database ambari;
Query OK,1 row affected (0.01sec)MariaDB [mysql]>use ambari;
Database changed
MariaDB [ambari]> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql;
Query OK,0 rows affected (0.01sec)Query OK,0 rows affected (0.00sec)Query OK,0 rows affected (0.00sec)Query OK,0 rows affected (0.06sec)Query OK,0 rows affected (0.00sec)
Statement prepared
... ...


4，配置ambari server


执行命令




ambari-server setup


回答选择项，其中JDK选择“”Custom“”，给出系统安装目录，数据库一定要选择高级配置，指定mariadb数据库和用户，本人系统中详细过程如下：




Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is'disabled'Customize user accountfor ambari-server daemon [y/n] (n)?n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 1To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)? ^C
Aborting ... Keyboard Interrupt.
[root@ep-bd01 downloads]# ambari-server setup
Using python/usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is'disabled'Customize user accountfor ambari-server daemon [y/n] (n)?n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 2WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are requiredfor configuring Kerberos security. If you plan to use Kerberos,please makesure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME:^C
Aborting ... Keyboard Interrupt.
[root@ep-bd01 downloads]# echo$JAVA_HOME/usr/java/jdk1.8.0_181-amd64
[root@ep-bd01 downloads]# $JAVA_HOME/bin/java -version
java version"1.8.0_181"Java(TM) SE Runtime Environment (build1.8.0_181-b13)
Java HotSpot(TM)64-Bit Server VM (build 25.181-b13, mixed mode)
[root@ep-bd01 downloads]# ambari-server setup
Using python/usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is'disabled'Customize user accountfor ambari-server daemon [y/n] (n)?n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 2WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are requiredfor configuring Kerberos security. If you plan to use Kerberos,please makesure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME:^C
Aborting ... Keyboard Interrupt.
[root@ep-bd01 downloads]# ambari-server setup
Using python/usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is'disabled'Customize user accountfor ambari-server daemon [y/n] (n)?n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 1To download the Oracle JDK and the Java Cryptography Extension (JCE) Policy Files you must accept the license terms found at http://www.oracle.com/technetwork/java/javase/terms/license/index.html and not accepting will cancel the Ambari Server setup and you must install the JDK and JCE files manually.
Do you accept the Oracle Binary Code License Agreement [y/n] (y)?y
Downloading JDK from http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz to /var/lib/ambari-server/resources/jdk-8u112-linux-x64.tar.gz
ERROR: Exiting with exit code 1.
REASON: Downloading or installing JDK failed:'Fatal exception:  Failed to download JDK: <urlopen error [Errno -2] Name or service not known>. Please check that the JDK is available at http://public-repo-1.hortonworks.com/ARTIFACTS/jdk-8u112-linux-x64.tar.gz. Also you may specify JDK file location in local filesystem using --jdk-location command line argument., exit code 1'. Exiting.
[root@ep-bd01 downloads]# ambari-server setup
Using python/usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is'disabled'Customize user accountfor ambari-server daemon [y/n] (n)?n
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8[2] Custom JDK==============================================================================Enter choice (1): 2WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are requiredfor configuring Kerberos security. If you plan to use Kerberos,please makesure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME:/usr/java/jdk1.8.0_181-amd64
Validating JDK on Ambari Server...done.
Check JDK versionforAmbari Server...
JDK version found:8Minimum JDK version is8 for Ambari. Skipping to setup different JDK forAmbari Server.
Checking GPL software agreement...
GPL Licensefor LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)?n
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)?y
Configuring database...==============================================================================Choose one of the following options:
[1] -PostgreSQL (Embedded)
[2] -Oracle
[3] - MySQL /MariaDB
[4] -PostgreSQL
[5] -Microsoft SQL Server (Tech Preview)
[6] -SQL Anywhere
[7] -BDB==============================================================================Enter choice (1): 3Hostname (localhost): ep-bd01
Port (3306):
Database name (ambari): ambari
Username (ambari): ambari
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...
Should ambari use existing default jdbc/usr/share/java/mysql-connector-java.jar [y/n] (y)?y
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema:/var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)?y
Extracting system views...
ambari-admin-2.7.0.0.897.jar
....
Ambari repofile doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server'setup' completed successfully.


5，命令行方式设置mysql数据库连接库，用于oozie和ranger连接mariadb时使用（上面设置后不起作用，必须如下操作）：




ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar


6，配置ambari-server自启动，启动ambari-server




[root@ep-bd01 downloads]# systemctl enable ambari-server
[root@ep-bd01 downloads]# systemctl start ambari-server
[root@ep-bd01 downloads]# systemctl status ambari-server
● ambari-server.service - LSB: ambari-server daemonLoaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled)Active: active (running) since Tue2018-08-14 11:06:16CST; 2min 36s agoDocs:man:systemd-sysv-generator(8)Process:323056 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS)CGroup:/system.slice/ambari-server.service└─323080 /usr/java/jdk1.8.0_181-amd64/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -...Aug14 11:05:58 ep-bd01 ambari-server[323056]: Organizing resource files at /var/lib/ambari-server/resources...
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Ambari database consistency check started...
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server PID at: /var/run/ambari-server/ambari-server.pid
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server out at: /var/log/ambari-server/ambari-server.out
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server log at: /var/log/ambari-server/ambari-server.log
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Waiting forserver start.....................
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Server started listening on 8080Aug14 11:06:16 ep-bd01 ambari-server[323056]: DB configs consistency check: no errors and warnings were found.
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Ambari Server 'start'completed successfully.
Aug14 11:06:16 ep-bd01 systemd[1]: Started LSB: ambari-server daemon.systemctl status ambari-server
● ambari-server.service - LSB: ambari-server daemonLoaded: loaded (/etc/rc.d/init.d/ambari-server; bad; vendor preset: disabled)Active: active (running) since Tue2018-08-14 11:06:16CST; 2min 36s agoDocs:man:systemd-sysv-generator(8)Process:323056 ExecStart=/etc/rc.d/init.d/ambari-server start (code=exited, status=0/SUCCESS)CGroup:/system.slice/ambari-server.service└─323080 /usr/java/jdk1.8.0_181-amd64/bin/java -server -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:-UseGCOverheadLimit -XX:CMSInitiatingOccupancyFraction=60 -XX:+CMSClassUnloadingEnabled -Dsun.zip.disableMemoryMapping=true -...Aug14 11:05:58 ep-bd01 ambari-server[323056]: Organizing resource files at /var/lib/ambari-server/resources...
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Ambari database consistency check started...
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server PID at: /var/run/ambari-server/ambari-server.pid
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server out at: /var/log/ambari-server/ambari-server.out
Aug14 11:05:58 ep-bd01 ambari-server[323056]: Server log at: /var/log/ambari-server/ambari-server.log
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Waiting forserver start.....................
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Server started listening on 8080Aug14 11:06:16 ep-bd01 ambari-server[323056]: DB configs consistency check: no errors and warnings were found.
Aug14 11:06:16 ep-bd01 ambari-server[323056]: Ambari Server 'start'completed successfully.
Aug14 11:06:16 ep-bd01 systemd[1]: Started LSB: ambari-server daemon.


ok，Ambari-server安装完成。浏览器访问ambari-server服务：




http://ep-bd01:8080


默认用户名/密码为：admin/admin，登陆后界面如下：




五、所有主机节点，安装ambari-agent，并配置自启动




yum install ambari-agent -y systemctl enable ambari-agent systemctl restart ambari-agent && systemctl status ambari-agent




转载于:https://www.cnblogs.com/dajianshi/p/9455980.html
												


											

基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库01 —— 准备环境，搭建本地仓库，安装ambari...相关推荐
	

    
								
  1. Java_JDK19.0.2_Ubuntu18.04中配合海康工业相机SDK环境搭建
		
    Java_JDK19.0.2_Ubuntu18.04中配合海康工业相机SDK环境搭建 简介 1. JDK环境搭建 JDK下载 JDK环境安装 2. 工业相机环境SDK安装 简介 本文主要介绍在ubun ...
    
		
    2. 
						
  2. 基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库02 ——使用ambari-server安装HDP...
		
    本文记录使用ambari-server安装HDP的过程,对比于使用cloudera-manager安装CDH,不得不说ambari的易用性差的比较多~_~,需要用户介入的过程较多,或者说可定制性更高. ...
    
		
    3. 
						
  3. 基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】搭建HAWQ数据仓库04 —— 安装HAWQ插件PXF3.3.0.0...
		
    一. 安装PXF3.3.0.0,这里所安装的pxf的包文件都包含在apache-hawq-rpm-2.3.0.0-incubating.tar.gz里面 下面步骤都是以root身份执行 这里注意,px ...
    
		
    4. 
						
  4. 机器学习实战:基于Scikit-Learn.Keras和TensorFlow(原书第2版) 奥雷利安·杰龙——环境搭建anaconda
		
    这个学期开了一门课,机器学习 然后实验部分用的是这本教材:机器学习实战:基于Scikit-Learn.Keras和TensorFlow 奥雷利安·杰龙,第几版的就不知道了. 用最新的机器学习库,一定会 ...
    
		
    5. 
						
  5. SpringBoot2.0基础案例(01)：环境搭建和RestFul风格接口
		
    一.SpringBoot 框架的特点 SpringBoot2.0 特点 1)SpringBoot继承了Spring优秀的基因,上手难度小 2)简化配置,提供各种默认配置来简化项目配置 3)内嵌式容器简 ...
    
		
    6. 
						
  6. 基于【CentOS-7+ Ambari 2.7.0 + HDP 3.0】HAWQ数据仓库 使用之 gpfdist协议
		
    一.HAWQ基本安装自带gpfdist协议 gpfdist是HAWQ支持的外部表访问协议之一,这是hawq自带的一个简单的集成http服务命令. 在我的前述安装hawq之后,gpfdist命令位于ha ...
    
		
    7. 
						
  7. ZigBee 3.0实战教程-Silicon Labs EFR32+EmberZnet-2-04：开发环境搭建
		
    [ 源码. 文档. 软件. 硬件. 技术交流. 技术支持, 入口见 文末] [所有相关 IDE. SDK和 例程源码均可从群文件 免费获取, 免安装, 解压即用] 持续更新中,欢迎关注! 一.解压资源 ...
    
		
    8. 
						
  8. .net core 2.0学习笔记（一）：开发运行环境搭建
		
    期待已久的.net core 2.0终于发布了!大家等的花儿都谢了. 不过比预期提前了一个多月,这在微软历史上还真的不多见.按照历史经验看,2.0版本应该比较靠谱,我猜这也是社区非常火爆的原因吧.下面 ...
    
		
    9. 
						
  9. 从0到1：CTFer成长之路docker环境搭建
		
    1. 安装 docker 使用官方安装脚本安装 sudo curl -fsSL https://get.docker.com | bash -s docker --mirror Aliyun 也可以使 ...
    
		
    10. 
		


					

最新文章
	

    
						
  1. 为什么 Redis 要比 Memcached 更火？
		
    2. 
						
  2. dateframe行列插入和删除操作
		
    3. 
						
  3. log4j 禁止类输出日志_log4j 2过滤spring日志遇到的问题
		
    4. 
						
  4. Android笔记(七十三) Android权限问题整理 非常全面
		
    5. 
						
  5. 接口测试工具--apipost脚本讲解
		
    6. 
						
  6. 50、BGP配置实验之社团属性community
		
    7. 
						
  7. 1.8 Linux用户与用户组文件权限
		
    8. 
						
  8. java多线程同步与测试_Java多线程同步Synchronized深入解析
		
    9. 
						
  9. win10计算机管理字体糊,win10字体发虚模糊正确解决方法(5个方法)
		
    10. 
						
  10. 中国地图着色问题c语言,中国地图四色染色问题.doc
		
    11. 
						
  11. SpriteKit在iOS8和OSX10.10中的新特性(强悍来袭)
		
    12. 
						
  12. 百度开放平台中的万年历被提取出来了
		
    13. 
						
  13. 文件输入流和文件输出流
		
    14. 
						
  14. 微信小程序项目创建步骤
		
    15. 
						
  15. java通过SMS短信平台实现发短信的功能
		
    16. 
						
  16. VBA word 常用代码及注释
		
    17. 
						
  17. 超级玛丽游戏（C语言）
		
    18. 
						
  18. excel添加自定义名称
		
    19. 
						
  19. [力扣刷题总结](双指针篇)
		
    20. 
						
  20. 计算机中丢失hvdailt.dll,修复hvdailt.dll
		
    21. 
		

	


热门文章
	

    
									
  1. 对云风 cstring  第二次解析
			
    2. 
						
  2. Enterprise Library 4 数据访问应用程序块
			
    3. 
						
  3. ROS_Kinetic     ubuntu 16.04
			
    4. 
						
  4. 经典算法书籍推荐以及算法书排行【算法四库全书】
			
    5. 
						
  5. Spring Boot集成Swagger导入YApi@无界编程
			
    6. 
						
  6. PHP版UTF-8文件BOM自动检测移除程序
			
    7. 
						
  7. 《Effective C#中文版：改善C#程序的50种方法》读书笔记
			
    8. 
						
  8. C# 2.0 的partial
			
    9. 
						
  9. ASP.NET 2.0数据处理之高级分页/排序
			
    10. 
						
  10. Ruby的Singleton method
			
    11.