hadoop2.4.1结合hbase0.96.2
接上:http://onlyoulinux.blog.51cto.com/7941460/1554951
上文说到用hadoop2.4.1分布式结合hase0.94.23出现大量的报错,没能解决,最后用新版本hbase0.96.2同样的配置没遇到报错,个人感觉是版本兼容性问题,报错内容如下:
2014年 09月 22日 星期一 19:56:03 CST Starting master on nn
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 5525
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 5525
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
2014-09-22 19:56:04,681 INFO org.apache.hadoop.hbase.util.VersionInfo: HBase 0.94.23
2014-09-22 19:56:04,681 INFO org.apache.hadoop.hbase.util.VersionInfo: Subversion git://asf911.gq1.ygridcore.net/home/jenkins/jenkin
s-slave/workspace/HBase-0.94.23 -r f42302b28aceaab773b15f234aa8718fff7eea3c
2014-09-22 19:56:04,681 INFO org.apache.hadoop.hbase.util.VersionInfo: Compiled by jenkins on Wed Aug 27 00:54:09 UTC 2014
2014-09-22 19:56:05,072 DEBUG org.apache.hadoop.hbase.master.HMaster: Set serverside HConnection retries=140
2014-09-22 19:56:05,852 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,853 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,854 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,855 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,856 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,857 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,858 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,858 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,859 INFO org.apache.hadoop.ipc.HBaseServer: Starting Thread-1
2014-09-22 19:56:05,863 INFO org.apache.hadoop.ipc.HBaseServer: Starting IPC Server listener on 60000
2014-09-22 19:56:05,889 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HMaster, port=60000
2014-09-22 19:56:06,510 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 3210@nn
2014-09-22 19:56:06,572 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012
17:52 GMT
2014-09-22 19:56:06,572 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=nn
2014-09-22 19:56:06,572 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_67
2014-09-22 19:56:06,572 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2014-09-22 19:56:06,572 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/java/jdk1.7.0_67/jre
"logs/hbase-root-master-nn.log" 165L, 30411C 1,1 顶端
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1206)
at org.apache.hadoop.ipc.Client.call(Client.java:1050)
... 18 more
2014-09-22 19:56:18,228 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2014-09-22 19:56:18,228 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads
2014-09-22 19:56:18,228 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60000
2014-09-22 19:56:18,231 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: exiting
2014-09-22 19:56:18,231 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: exiting
2014-09-22 19:56:18,231 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: exiting
2014-09-22 19:56:18,232 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: exiting
2014-09-22 19:56:18,232 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: exiting
2014-09-22 19:56:18,232 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: exiting
2014-09-22 19:56:18,233 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: exiting
2014-09-22 19:56:18,233 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: exiting
2014-09-22 19:56:18,233 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: exiting
2014-09-22 19:56:18,234 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: exiting
2014-09-22 19:56:18,234 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60000: exiting
2014-09-22 19:56:18,234 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60000: exiting
2014-09-22 19:56:18,234 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60000: exiting
2014-09-22 19:56:18,235 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60000
2014-09-22 19:56:18,240 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2014-09-22 19:56:18,240 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2014-09-22 19:56:18,241 INFO org.apache.hadoop.hbase.master.HMaster: Stopping infoServer
2014-09-22 19:56:18,247 INFO org.mortbay.log: Stopped SelectChannelConnector@0.0.0.0:60010
2014-09-22 19:56:18,278 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2014-09-22 19:56:18,278 INFO org.apache.zookeeper.ZooKeeper: Session: 0x1489d3757740000 closed
2014-09-22 19:56:18,278 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exiting
2014-09-22 19:56:18,278 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
java.lang.RuntimeException: HMaster Aborted
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:160)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129)
2014-09-22 20:35:55,320 INFO org.mortbay.log: jetty-6.1.26
2014-09-22 20:35:57,557 INFO org.mortbay.log: Started SelectChannelConnector@0.0.0.0:60010
2014-09-22 20:35:57,645 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Deleting ZNode for /hbase/backup-masters/nn,60000,1
411389351537 from backup master directory
2014-09-22 20:35:58,013 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Node /hbase/backup-masters/nn,60000,14113893515
37 already deleted, and this is not a retry
2014-09-22 20:35:58,013 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=nn,60000,1411389351537
2014-09-22 20:36:00,237 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 0 tim
e(s).
2014-09-22 20:36:01,239 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 1 tim
e(s).
2014-09-22 20:36:02,312 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 2 tim
e(s).
2014-09-22 20:36:03,314 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 3 tim
e(s).
2014-09-22 20:36:04,320 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 4 tim
e(s).
2014-09-22 20:36:05,321 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 5 tim
e(s).
2014-09-22 20:36:06,322 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 6 tim
e(s).
2014-09-22 20:36:07,323 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 7 tim
e(s).
2014-09-22 20:36:08,325 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 8 tim
e(s).
2014-09-22 20:36:09,389 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: localhost/127.0.0.1:9000. Already tried 9 tim
e(s).
2014-09-22 20:36:09,395 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.net.ConnectException: Call to localhost/127.0.0.1:9000 failed on connection exception: java.net.ConnectException: 拒绝连接
at org.apache.hadoop.ipc.Client.wrapException(Client.java:1099)
at org.apache.hadoop.ipc.Client.call(Client.java:1075)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at com.sun.proxy.$Proxy10.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379)
at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:238)
at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:203)
at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386)
下面是我的配置:
> hbase-env.sh
export JAVA_HOME=/usr/java/jdk1.7.0_67/
export HBASE_HOME=/usr/local/hbase-0.96.2-hadoop2
export HADOOP_HOME=/usr/local/hadoop-2.4.1/
export HBASE_MANAGES_ZK=false
> regionservers
[root@nn hbase-0.96.2-hadoop2]# cat conf/regionservers
nn
sn
dn1
> hbase-site.xml
[root@nn hbase-0.96.2-hadoop2]# cat conf/hbase-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-->
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://nn:9000/tmpdir</value>
<description>The directory shared by RegionServers.</description>
</property>
<property>
<name>dfs.replication</name>
<value>2</value>
<description>The replication count for HLog and HFile storage. Should not be greater than HDFS datanode count.</description>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>nn,sn,dn1</value>
</property>
<property>
<name>hbase.zookeeper.property.datadir</name>
<value>/tmpdir/zookeeperdata/</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
<description>Property from ZooKeeper's config zoo.cfg.The port at which the clients will connect.
</description>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.hregion.memstore.flush.size</name>
<value>5242880</value>
</property>
<property>
<name>hbase.hregion.memstore.flush.size</name>
<value>5242880</value>
</property>
<property>
<name>hbase.master</name>
<value>master:60000</value>
</property>
</configuration>
启动并验证:
简单的bash shell测试:
[root@sn test]# hbase shell
2014-09-26 16:22:26,178 INFO [main] Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.96.2-hadoop2, r1581096, Mon Mar 24 16:03:18 PDT 2014
hbase(main):001:0> list
TABLE
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
0 row(s) in 6.7970 seconds
=> []
hbase(main):002:0> create
create create_namespace
hbase(main):002:0> create 'test','stu_id','stu_xx'
0 row(s) in 18.0590 seconds
=> Hbase::Table - test
hbase(main):003:0> put 'test','1','stu_id:name','jaybing'
0 row(s) in 3.2750 seconds
hbase(main):004:0> scan 'test'
ROW COLUMN+CELL
1 column=stu_id:name, timestamp=1411720116252, value=jaybing
1 row(s) in 0.2280 seconds
hbase(main):005:0> list
TABLE
test
1 row(s) in 0.2460 seconds
=> ["test"]
hbase(main):006:0>
转载于:https://blog.51cto.com/onlyoulinux/1558699
hadoop2.4.1结合hbase0.96.2相关推荐
- Flume1.5.0的安装、部署、简单应用(含伪分布式、与hadoop2.2.0、hbase0.96的案例)
原文地址:http://www.cnblogs.com/lion.net/p/3903197.html 目录: 一.什么是Flume? 1)flume的特点 2)flume的可靠性 3)flume的可 ...
- oracle11hadoop混搭,【甘道夫】Hadoop2.2.0环境使用Sqoop-1.4.4将Oracle11g数据导入HBase0.96,并自动生成组合行键...
目的: 使用Sqoop将Oracle中的数据导入到HBase中,并自动生成组合行键! 环境: Hadoop2.2.0 Hbase0.96 sqoop-1.4.4.bin__hadoop-2.0.4-a ...
- Flume-0.9.4数据插入HBase-0.96
来自:http://blog.csdn.net/iam333/article/details/18770977 最近由于业务需要,需要将flume的数据插入HBase-0.96,利用flume的实时日 ...
- Hbase0.96 MVCC Lock 知识梳理
HBASE0.96 MVCC 写入的时候 每个Region包含一个Memstore,维护一个MultiVersionConsistencyControl对象 w = mvcc.beginMemstor ...
- Hbase读写数据过程详解(hbase0.96版本之前与hbase0.96版本之后对比)
@Author : Spinach | GHB @Link : http://blog.csdn.net/bocai8058 文章目录 HBase架构图 -ROOT-和.META.结构 -ROOT- ...
- hbase0.96—+版本的endpoint
HBase Coprocessor 之 endpiont(hbase 0.96.0) 分类: hbase 2014-04-22 15:23 1661人阅读 评论(0) 收藏 举报 hbase ...
- Flume-ng配置
1)简介 Flume是一个分布式.可靠.和高可用的海量日志聚合的系统,支持在系统中定制各类数据发送方,用于收集数据:同时,Flume提供对数据进行简单处理,并写到各种数据接受方(可定制)的能力. 设计 ...
- Flume+Kafka+Strom基于伪分布式环境的结合使用
--------------------------------------- 博文作者:迦壹 博客地址:Flume+Kafka+Strom基于伪分布式环境的结合使用 转载声明:可以转载, 但必须以超 ...
- 1-0 三台PC集群的搭建(noHA)
1.部署 搭建方案(no_HA): project hadoop hadoop Hbase zookeeper 主机 master1 Namenode datanode(NM) HMaster HRe ...
最新文章
- 深入理解Spark:核心思想与源码分析. 3.9 启动测量系统MetricsSystem
- 三天打工生活终于结束了
- Mybatis实现物理分页
- powerdesigner mysql 反引号_PowerDesigner实用技巧小结 及 导出word,想字段顺序跟模型中一致,如何设置...
- pythonmessage用法_django 消息框架 message使用详解
- [Linux] 权限与指令间的关系
- 苹果推出36期免息分期?每月88元就能用上iPhone 13
- 离散数学及其应用 前言
- socket 10106问题解决日记
- 设置Notes暗黑模式
- JZOJ 6310.glo【LIS】【线段树】
- matlab分栏画图,Tkinter编程应知应会(19)-分栏窗口控件PanedWindow
- 如何成为一个很厉害的人(时间之尺)
- python注释第一行_python第一行注释是什么意思?
- 极简设计模式 - UML类图几种线条
- 学PS平面设计前,你需要先了解这些
- linux下qt浏览word文件内容,Qt获取office文件内容
- 沟通和编程一样,也是一门艺术系列1(最佳的沟通态度)
- 记录mysql查询过去十二个月中每个月的数据情况(含本月)
- stm8 ADC中断采集