CentOS系统下Hadoop 2.4.1集群安装配置（简易版）

2024-04-07 23:31:17

安装配置

1、软件下载

JDK下载:jdk-7u65-linux-i586.tar.gz

http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

Hadoop下载：hadoop-2.4.1.tar.gz

http://www.apache.org/dyn/closer.cgi/hadoop/common/

2、/etc/hosts配置

[html] view plaincopyprint?

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
<strong><span style="color:#ff0000;">192.168.1.2 Master.Hadoop
192.168.1.3 Slave1.Hadoop</span></strong>

3、/etc/profile配置

[html] view plaincopyprint?

export JAVA_HOME=/usr/java/jrockit-jdk1.6.0_45-R28.2.7-4.1.0
export CLASSPATH=.:$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export PATH=$PATH:$JAVA_HOME/bin:$JAVA_HOME/jre/bin
export HADOOP_HOME=/usr/hadoop
export HADOOP_HOME_WARN_SUPPRESS=1
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin

4、~/etc/hadoop/core-site.xml配置

[html] view plaincopyprint?

<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://Master.Hadoop:9000</value>
<description>
Where to find the Hadoop Filesystem through the network.
Note 9000 is not the default port.
(This is slightly changed from previous versions which didnt have "hdfs")
</description>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/usr/hadoop/tmp</value>
</property>
</configuration>

5、~/etc/hadoop/mapred-site.xml配置

[html] view plaincopyprint?

<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>

6、etc/hadoop/yarn-site.xml配置

[html] view plaincopyprint?

<configuration>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>Master.Hadoop:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>Master.Hadoop:8031</value>
</property>
<property>
<name>yarn.resourcemanager.address</name>
<value>Master.Hadoop:8032</value>
</property>
<property>
<name>yarn.resourcemanager.admin.address</name>
<value>Master.Hadoop:8033</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.address</name>
<value>Master.Hadoop:8088</value>
</property>
<property>
<name>yarn.resourcemanager.webapp.https.address</name>
<value>Master.Hadoop:8090</value>
</property>
<property>
<name>yarn.nodemanager.local-dirs</name>
<value>${hadoop.tmp.dir}/nodemanager/local</value>
<description>the local directories used by the nodemanager</description>
</property>
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>${hadoop.tmp.dir}/nodemanager/remote</value>
<description>directory on hdfs where the application logs are moved to </description>
</property>
<property>
<name>yarn.nodemanager.log-dirs</name>
<value>${hadoop.tmp.dir}/nodemanager/logs</value>
<description>the directories used by Nodemanagers as log directories</description>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
<description>shuffle service that needs to be set for Map Reduce to run </description>
</property>
<property>
<name>mapreduce.jobhistory.address</name>
<value>Master.Hadoop:10020</value>
</property>
<property>
<name>mapreduce.jobhistory.webapp.address</name>
<value>Master.Hadoop:19888</value>
</property>
</configuration>

7、~/etc/hadoop/hdfs-site.xml配置

[html] view plaincopyprint?

<configuration>
<property>
<name>dfs.permissions.superusergroup</name>
<value>root</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

启动与验证

1、格式化HDFS文件系统

hadoop namenode -format

2、启动Hadoop

启动前关闭集群中所有机器的防火墙

service iptables stop

启动命令

start-all.sh

3、验证Hadoop

方式一：jps

方式二：hadoop dfsadmin -report

转载于:https://www.cnblogs.com/gw811/p/4077302.html

CentOS系统下Hadoop 2.4.1集群安装配置（简易版）相关推荐

Hadoop 2.5.1集群安装配置
本文的安装只涉及了hadoop-common.hadoop-hdfs.hadoop-mapreduce和hadoop-yarn,并不包含HBase.Hive和Pig等. http://blog.csd ...
centos 7下Hadoop 2.7.2 伪分布式安装
centos 7 下Hadoop 2.7.2 伪分布式安装,安装jdk,免密匙登录,配置mapreduce,配置YARN.详细步骤如下: 1.0 安装JDK 1.1 查看是否安装了openjdk [l ...
一步步教你Hadoop多节点集群安装配置
一步步教你Hadoop多节点集群安装配置 1.集群部署介绍 1.1 Hadoop简介 Hadoop是Apache软件基金会旗下的一个开源分布式计算平台.以Hadoop分布式文件系统HDFS(Hado ...
原创：centos7.1下 ZooKeeper 集群安装配置+Python实战范例
centos7.1下 ZooKeeper 集群安装配置+Python实战范例下载:http://apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeepe ...
hadoop 2.4.1 集群安装一
配置主机名参考 Hadoop 1.2.1 集群安装一配置JDK环境参考Hadoop1.2.1集群安装二配置hadoop A:下载解压hadoop http://mirrors.cnnic.cn/a ...
RabbitMQ集群安装配置+HAproxy+Keepalived高可用
RabbitMQ集群安装配置+HAproxy+Keepalived高可用转自:https://www.linuxidc.com/Linux/2016-10/136492.htm rabbitmq 集 ...
ZooKeeper-3.3.4集群安装配置
"ZooKeeper-3.3.4集群安装配置": 关键词:zookeeper-3.3.4 集群安装配置 zookeeper是一个分布式开源框架,提供了协调分布式应用的基本服务, ...
Openpbs centos7集群安装配置心得
Openpbs centos7集群安装配置心得写在前面准备工作 1.安装虚拟机 2.创建虚拟机集群 SSH免密登陆网络环境配置 ssh免密登陆建立NFS共享目录关闭各节点防火墙和Selinu ...
ElasticSearch-2.0.0集群安装配置与API使用实践
ElasticSearch是基于全文搜索引擎库Lucene构建的分布式搜索引擎,我们可以直接使用ElasticSearch实现分布式搜索系统的搭建与使用,都知道,Lucene只是一个搜索框架,它提供了 ...

最新文章

热门文章