集群节点基于Hadoop集群的HBase集群的配置【2】
首先声明,我是一个菜鸟。一下文章中现出技术误导情况盖不负责
基于Hadoop集群的HBase集群的配置
1.准备工作
a.经已实现安装的Hadoop集群环境
b.所须要的软件包
zookeeper-3.4.4.tar.gz下载地址:
http://downloads.gotsoft.com/pub/apache/zookeeper/zookeeper-3.4.4/
hbase-0.94.6.1tar.gz下载地址:
http://labs.mop.com/apache-mirror/hbase/stable/hbase-0.94.6.1.tar.gz
2.独单安装的ZooKeeper集群,不基于Hbase集群管理
a.在Master节点上解压zookeeper
l ##提议用使hduser权限,免避后之现出SSH错误
hduser@master:/usr/local$sudo tar -zxvfzookeeper-3.4.4.tar.gz
l ##server前面的数字是zooKeeper识别时后的Id,须要与前面的置设保持一致
hduser@master:/usr/local$tvi /zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=5
syncLimit=2
dataDir=/data/zookeeper
clientPort=2181
server.1=master:2888:3888
server.2=slave1:2888:3888
server.3=slave2:2888:3888
root@master:/usr/local# chown -Rhduser:hadoop zookeeper
3.将zookeeper复制到其他呆板
hduser@master:/usr/local$scp -r zookeeper-3.4.5hduser@slave1:/home/hduser
hduser@master:/usr/local$ scp -rzookeeper-3.4.5 hduser@slave2:/home/hduser
hduser@slave1:~#sudomv zookeeper-3.4.5/ /usr/local/zookeeper-3.4.5
hduser@slave2:~$sudomv zookeeper-3.4.5/ /usr/local/zookeeper-3.4.5
4.在每台呆板配置的dataDir目录上面,创立myid文件标记前当机主
Master: $touch /data/zookeeper/myid
$echo "1" >/data/zookeeper/myid
Slave1: $mkdir -p /data/zookeeper
$touch /data/zookeeper/myid
$echo "3" > /data/zookeeper/myid
Slave2: $mkdir -p/data/zookeeper
$touch /data/zookeeper/myid
$echo "5" > /data/zookeeper/myid
5. 动启zookeeper集群
hduser@master:/usr/local/zookeeper/bin$./zkServer.sh start
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
【题问1】Starting zookeeper ... ./zkServer.sh: line 113:/data/zookeeper/zookeeper_server.pid: Permission denied
FAILED TO WRITE PID
因原是刚才/data/zookeeper是root权限,而不是hduser的,改修权限之~
【题问1处理】root@master:/# chown -Rhduser:hadoop /data
其他两个slave节点也一样~
然后题问处理:
hduser@master:/usr/local/zookeeper$ cd bin
hduser@master:/usr/local/zookeeper/bin$ ./zkServer.sh start
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
其他节点也样同动启zookeeper~
hduser@master:/usr/local/zookeeper/bin$
hduser@slave1:/usr/local/zookeeper/bin$./zkServer.sh status
JMX enabled by default
Using config: /usr/local/zookeeper/bin/../conf/zoo.cfg
【题问2】Error contacting service. It is probably notrunning.
1)查看zoopkeeper.out
r@232] - Unexpected exception,tries=0, connecting to slave1/192.168.123.12:2888
java.net.ConnectException: Connection refused
atjava.net.PlainSocketImpl.socketConnect(Native Method)
atjava.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
atjava.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
atjava.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
atjava.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:529)
root@master:/usr/local/zookeeper/bin#cat zookeeper.out
2013-04-27 00:22:37,595 [myid:] - INFO [main:QuorumPeerConfig@101] - Reading configuration from:/usr/local/zookeeper/bin/../conf/zoo.cfg
2013-04-27 00:22:37,615 [myid:] - ERROR[main:QuorumPeerMain@85] - Invalid config, exiting abnormally
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException:Error processing /usr/local/zookeeper/bin/../conf/zoo.cfg
at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:121)
atorg.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)
atorg.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by:java.lang.IllegalArgumentException: dataLogDir /home/hduser/zookeeper/log ismissing.
atorg.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:247)
atorg.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:117)
... 2 more
Invalid config, exiting abnormally
【错误2处理】没有将zookeeper复制到其他的节点!!!
没有在个每节点动启zookeeper
6.安装配置Hbase集群
a>动启HDFS集群例实,并创立目录
##hdfs://master01:54310/hbase【注意与hadoop,core-site.xml中的fs.default.name中的端口号保持一致,一统置设为54310;hbase-site.xml中的hbase:rootdir中的端口号】
hduser@master:/usr/local/hadoop$bin/hadoop fs -mkdir /hbase
Warning: $HADOOP_HOME is deprecated.
13/04/27 02:08:46 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 0 time(s).
13/04/27 02:08:47 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 1 time(s).
有些冷,有些凉,心中有些无奈,我一个人走在黑夜中,有些颤抖,身体瑟缩着,新也在抖动着,我看不清前方的路,何去何从,感觉迷茫,胸口有些闷,我环视了一下周围,无人的街头显得冷清,感到整个世界都要将我放弃。脚步彷徨之间,泪早已滴下……
13/04/27 02:08:48 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 2 time(s).
13/04/27 02:08:49 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 3 time(s).
13/04/27 02:08:50 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 4 time(s).
13/04/27 02:08:51 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 5 time(s).
13/04/27 02:08:52 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 6 time(s).
13/04/27 02:08:53 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 7 time(s).
13/04/27 02:08:54 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 8 time(s).
13/04/27 02:08:55 INFO ipc.Client: Retryingconnect to server: master/127.0.1.1:54310. Already tried 9 time(s).
Bad connection to FS. command aborted.exception: Call to master/127.0.1.1:54310 failed on connection exception:java.net.ConnectException: Connection refused
hduser@master:/usr/local/hadoop$
Call tomaster/127.0.1.1:54310 failed【改修localhost地址为127.0.0.1】
HBASE SHELL 错误NativeException: org.apache.hadoop.hbase.MasterNotRunningException:null
【处理步调:】
1、动启hadoop后,须要等一段时间,再开启hbase
2、去失落hadoop的安全模式:
hadoop dfsadmin -safemode leave
3、把/etc/hosts里的master01的IP为改服务器前当的IP
4、确认hbase的hbase-site.xml中
<name>hbase.rootdir</name>
<value>hdfs://localhost:8020/hbase</value>
与hadoop的core-site.xml中
<name>fs.default.name</name>
<value>hdfs://localhost:8020</value>
红字分部保持一致
5、确认hadoop版本是0.20.2,hbase版本是0.20.6
6、从新执行./start-hbase.sh之前,先kill失落前当的hbase和zookeeper程进
后之再执行:
$bin/hadoop fs -mkdir /hbase
b> 解压hbase-0.94.5.tar.gz,改修配置文件,正式安装HBase:
l #vi hbase-0.94.5/conf/hbase-env.sh
exportJAVA_HOME=/usr/java/jdk1.6.0_43
exportHBASE_CLASSPATH=$HBASE_HOME/
#exportHBASE_HEAPSIZE=128
#exportHBASE_OPTS="-XX:+UseConcMarkSweepGC"
exportHBASE_MANAGES_ZK=false
#vi hbase-0.94.5/conf/regionservers
slave1
slave2
l 将hbase-webapps目录改名为webapps
hduser@master:/usr/local/hbase-0.94.6$ mvhbase-webapps/ /webapps
l 检查HBase配置,复制到其他的slave节点:
hduser@master:/usr/local$ scp -rhbase-0.94.6 hduser@slave1:/home/hduser/
hduser@master:/usr/local$ scp -rhbase-0.94.6 hduser@slave2:/home/hduser/
l 动启HBase:
hduser@master:/usr/local$ ./hbase-0.94.6/bin/start-hbase.sh
starting master, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-master-master.out
slave2: starting regionserver, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-regionserver-slave2.out
slave1: starting regionserver, logging to/usr/local/hbase-0.94.6/bin/../logs/hbase-hduser-regionserver-slave1.out
l 查看关相程进:
hduser@master:/usr/local$ jps
3339 QuorumPeerMain
8210 JobTracker
7894 DataNode
9187 HMaster
8126 SecondaryNameNode
8459 TaskTracker
9328 Jps
7659 NameNode
hduser@master:/usr/local$
hduser@slave1:~$ jps
5416 TaskTracker
5879 Jps
5210 DataNode
3323 QuorumPeerMain
hduser@slave1:~$
hduser@slave2:/usr/local/hbase-0.94.6$ jps
5083 TaskTracker
5544 Jps
4877 DataNode
3042 QuorumPeerMain
【错误】13/04/27 05:47:21ERROR client.HConnectionManager$HConnectionImplementation: Check the valueconfigured in 'zookeeper.znode.parent'. There could be a mismatch with the oneconfigured in the master.
l 去失落hadoop的安全模式
hduser@master:/usr/local/hadoop$ bin/hadoopdfsadmin -safemode leave
Warning: $HADOOP_HOME is deprecated.
Safe mode is OFF
【处理方法】:http://blog.sina.com.cn/s/blog_718335510100zchp.html
l 检修HBase安装实现:
hbase(main):003:0> create'member','member_id','address','info'
0 row(s) in 1.6400 seconds
hbase(main):004:0> list
TABLE
member
1 row(s) in 0.0730 seconds
hbase(main):006:0* describe 'member'
DESCRIPTION ENABLED
{NAME => 'member', FAMILIES => [{NAME => 'address',true
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NON
E', REPLICATION_SCOPE => '0', VERSIONS => '3', COMP
RESSION => 'NONE', MIN_VERSIONS => '0', TTL =>'214
7483647', KEEP_DELETED_CELLS => 'false',BLOCKSIZE
=> '65536', IN_MEMORY => 'false', ENCODE_ON_DISK=>
'true', BLOCKCACHE => 'true'}, {NAME => 'info',DA
TA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NONE',
REPLICATION_SCOPE => '0', VERSIONS => '3',COMPRES
SION => 'NONE', MIN_VERSIONS => '0', TTL =>'214748
3647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE=>
'65536', IN_MEMORY => 'false', ENCODE_ON_DISK =>'t
rue', BLOCKCACHE => 'true'}, {NAME =>'member_id',
DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER =>'NONE
', REPLICATION_SCOPE => '0', VERSIONS => '3',COMPR
ESSION => 'NONE', MIN_VERSIONS => '0', TTL => '2147
483647', KEEP_DELETED_CELLS => 'false', BLOCKSIZE=
> '65536', IN_MEMORY => 'false', ENCODE_ON_DISK=>
'true', BLOCKCACHE =>'true'}]}
1 row(s) in 0.2170 seconds
文章结束给大家分享下程序员的一些笑话语录: 3G普不普及现在已经不是看终端了,而是看应用,有好的,便宜实用的应用,花1000多买个能用的智能手机应该不是什么难事。反过来说,你200元拿一个智能手机,没有好的应用,看个电影要几十元,也是没人用3G。
集群节点基于Hadoop集群的HBase集群的配置【2】相关推荐
- Hadoop-2.8.0集群搭建、hadoop源码编译和安装、host配置、ssh免密登录、hadoop配置文件中的参数配置参数总结、hadoop集群测试,安装过程中的常见错误
25. 集群搭建 25.1 HADOOP集群搭建 25.1.1集群简介 HADOOP集群具体来说包含两个集群:HDFS集群和YARN集群,两者逻辑上分离,但物理上常在一起 HDFS集群: 负责海量数据 ...
- Hadoop(六)搭建分布式HBase集群
接上一章节,在单机版的HBase中,HMaster,HRegionServer,Zookeeper都在一个JVM进程中运行,通过两个阶段来搭建.学习分布式的HBase,伪分布式和分布式. 一,伪分布式 ...
- 基于linux的oracle_rac实时应用集群研究,基于Linux的OracleRAC实时应用集群研究毕业设计论文(资料4)...
<基于Linux的OracleRAC实时应用集群研究毕业设计论文.doc>由会员分享,可免费在线阅读全文,更多与<基于Linux的OracleRAC实时应用集群研究毕业设计论文> ...
- 基于linux的oracle_rac实时应用集群研究,基于Linux的OracleRAC实时应用集群研究毕业设计论文...
<基于Linux的OracleRAC实时应用集群研究毕业设计论文.doc>由会员分享,可免费在线阅读全文,更多与<基于Linux的OracleRAC实时应用集群研究毕业设计论文> ...
- 小丸子学Hadoop系列之——部署Hbase集群
0.集群规划 主机名 ip地址 安装的软件 运行的进程 AI-OPT-HBS01 10.46.52.30 hadoop,hbase namenode,zkfc,resourcemanager AI-O ...
- hbase集群 数据写入_一种构建HBase集群全文索引方法,数据读取方法以及数据写入方法与流程...
本发明涉及HBase集群领域,尤其涉及一种构建HBase集群全文索引方法,数据读取方法以及数据写入方法. 背景技术: 随着云计算技术的不断发展,云计算技术不断落地成为支撑各行业信息技术发展的重要支柱. ...
- hadoop hive集群_失眠大数据专家,基于Hadoop不同版本搭建hive集群(附配置文件)
前言 本教程采用了两种方案 一种是hive-1.21版本,hadoop版本为hadoop2.6.5 还有一种是主要讲基于hadoop3.x hive的搭建 先来第一种 一.本地方式(内嵌derby) ...
- 基于Hadoop HA集群部署HBase HA集群(详细版)
文章目录 1.前言 2.ZooKeeper与Hadoop.HBase的关系 3.Hadoop与HBase的关系 4.架构资源规划 5.ZooKeeper集群设置 5.1 设置nn节点的zoo.conf ...
- Hadoop集群搭建Hbase集群
Habse介绍 HBase是一个分布式的.面向列的开源数据库,该技术来源于 Fay Chang 所撰写的Google论文"Bigtable:一个结构化数据的分布式存储系统".就像B ...
最新文章
- 币圈“水逆”,暴跌与高危漏洞
- python 在互联网应用是如此强大
- 性能测试工具比较:LoadRunner vs JMeter - 测试结果数据比较
- 原生线程池这么强大,Tomcat 为何还需扩展线程池?
- PHP7 新特性 简介
- LINQ语句之Select/Distinct和Count/Sum/Min/Max/Avg
- uploadify 3.1 的修改
- vue 文件转base64方法 base64转blob路径方法 a链接下载文件 form表单下载文件
- 【多元统计分析】05.多元统计的“三大分布”
- Linux网络编程之epoll模型
- WP手机升级WIN10被PIN码锁定
- 2022/8/8测试总结
- cin cin.get cin.getlin
- 求四边形最大内接矩形,一种不规则多边形的最大内接矩形的快速近似求解方法与流程...
- 我所认知的世界,不是Fragmention,而是Think
- 终身学习C++__小胡亲测
- gfsj(getit)
- 1.11.ARM的37个寄存器详解
- Neuraltalk2
- Java找出游戏的获胜者leetcode_1823
热门文章
- Lambda 表达式与 Stream
- jenkins pipeline之语法篇
- 多个 小程序_最简单的小程序制作方法,不会代码也能撸一个
- pg批量插入_在PostgreSQL中批量/批量更新/提升
- pillow支持python 2和python 3_python3读取图像并可视化的方法(PIL/Pillow、opencv/cv2)...
- 蓝牙广播错误码3_蓝牙简介—物理层(PHY)
- 谁不想写出干净的代码?7 个技巧,让你的同事爱上你的代码
- php判断目录是否有写的权限,PHP版目录权限检测
- 中考可用计算机吗,中考能用什么计算器 可以带电子手表吗
- QPainter的使用