说说单节点集群里安装hive、3\5节点集群里安装hive的诡异区别(版本搭配)
这几天,无意之间,被这件事情给迷惑,不解!先暂时贴于此,以后再解决!
详细问题如下:
在hive的安装目录下(我这里是 /home/hadoop/app/hive-1.2.1),hive的安装目录的lib下(我这里是/home/hadoop/app/hive-1.2.1/lib)存放了mysql-connector-java-5.1.21.jar。
我的mysql,是用root用户安装的,在/home/hadoop/app目录,所以,启动也得在此目录下。
对于djt002,我的mysql是root用户安装的,在/usr/java目录,所以,启动也得在此目录下。service mysqld start
在单节点集群里。主机名为weekedn110。
<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://weekend110:3306/hive?createDatabaseIfNotExist=true</value><description>JDBC connect string for a JDBC metastore</description> </property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description> </property><property><name>javax.jdo.option.ConnectionUserName</name><value>hive</value><description>username to use against metastore database</description> </property><property><name>javax.jdo.option.ConnectionPassword</name><value>hive</value><description>password to use against metastore database</description> </property>
[root@weekend110 app]# service mysqld start
Starting mysqld: [ OK ]
[root@weekend110 app]# su hadoop
[hadoop@weekend110 app]$ cd hive-0.12.0/
[hadoop@weekend110 hive-0.12.0]$ bin/hive
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.input.dir.recursive is deprecated. Instead, use mapreduce.input.fileinputformat.input.dir.recursive
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.max.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.min.split.size is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.min.split.size.per.rack is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.rack
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.min.split.size.per.node is deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize.per.node
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.reduce.tasks is deprecated. Instead, use mapreduce.job.reduces
16/11/02 09:00:20 INFO Configuration.deprecation: mapred.reduce.tasks.speculative.execution is deprecated. Instead, use mapreduce.reduce.speculative
Logging initialized using configuration in jar:file:/home/hadoop/app/hive-0.12.0/lib/hive-common-0.12.0.jar!/hive-log4j.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/app/hadoop-2.4.1/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/app/hive-0.12.0/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J:
可以看出,这是单节点集群安装hive。直接这样既可。
但是,对于3节点集群。我测试过了,在HadoopMaster或HadoopSlave1或HadoopSlave2上,随便一台安装hive,以及同时启动hadoop进程。都是报如下错误。
[root@HadoopMaster app]# service mysqld status
mysqld (pid 3041) is running...
[root@HadoopMaster app]# su hadoop
[hadoop@HadoopMaster app]$ cd hive-1.2.1/
[hadoop@HadoopMaster hive-1.2.1]$ bin/hive
Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 14 more
Caused by: MetaException(message:Got exception: java.io.IOException No FileSystem for scheme: hfds)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.logAndThrowMetaException(MetaStoreUtils.java:1213)
at org.apache.hadoop.hive.metastore.Warehouse.getFs(Warehouse.java:106)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:140)
at org.apache.hadoop.hive.metastore.Warehouse.getDnsPath(Warehouse.java:146)
at org.apache.hadoop.hive.metastore.Warehouse.getWhRoot(Warehouse.java:159)
at org.apache.hadoop.hive.metastore.Warehouse.getDefaultDatabasePath(Warehouse.java:177)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB_core(HiveMetaStore.java:600)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:620)
at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:461)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:66)
at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:72)
at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:5762)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:199)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
... 19 more
[hadoop@HadoopMaster hive-1.2.1]$
[hadoop@HadoopSlave1 hive-1.2.1]$ bin/hive
Logging initialized using configuration in jar:file:/home/hadoop/app/hive-1.2.1/lib/hive-common-1.2.1.jar!/hive-log4j.properties
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
... 8 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 14 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
对此,真心,感觉到困惑。!!!
解决办法: 其实是版本的问题。hadoop-2.6.0.tar.gz的用户与hive-1.0.0搭配!
hive常见问题解决干货大全 (问题7)
再谈hive-1.0.0与hive-1.2.1到JDBC编程忽略细节问题
转载于:https://www.cnblogs.com/zlslch/p/6021805.html
说说单节点集群里安装hive、3\5节点集群里安装hive的诡异区别(版本搭配)相关推荐
- Cloudera Manager安装之利用parcels方式安装3或4节点集群(包含最新稳定版本或指定版本的安装)(添加服务)(CentOS6.5)(五)...
参考博客 Cloudera Manager安装之利用parcels方式安装单节点集群 Cloudera Manager安装之Cloudera Manager 5.3.X安装(三)(tar方式.rpm ...
- Cloudera Manager安装之利用parcels方式(在线或离线)安装3或4节点集群(包含最新稳定版本或指定版本的安装)(添加服务)(Ubuntu14.04)(五)...
如果大家,在启动的时候,比如遇到如下问题,则 明明已经授权了啊,怎么被拒绝,纳尼??? 解决办法 然后,再来这样,就可以了. 注意,在此之前. ubuntucmbigdata1机器上,则需要执行 bi ...
- Kubeadm安装高可用的K8S集群--多master单node
Kubeadm安装高可用的K8S集群–多master单node master1 IP 192.168.1.180/24 OS Centos7.6 master2 IP 192.168.1.181/24 ...
- [Hadoop集群模式安装与配置全过程]红帽子RatHat6下hadoop集群模式安装(3个节点,master,slave1,slave2)
红帽子RatHat6下hadoop集群模式安装(3个节点,master,slave1,slave2) 一.学习目标: 使用红帽子RatHat6在虚拟机上搭建hadoop集群,包含3个节点,体验集群分布 ...
- 二进制安装k8s v1.22.8集群
一.环境规划 1.1 服务器环境 K8S集群角色 Ip 主机名 安装的组件 控制节点 192.168.10.162 k8s-master01 etcd.docker.kube-apiserver.ku ...
- Java中高级核心知识全面解析——Redis(集群【概述{主从复制、哨兵、集群化}、数据分区方案、节点通信机制、数据结构简析】)5
目录 一.[集群]入门实践教程 1.Redis 集群概述 1)Redis 主从复制 2)Redis 哨兵 3)Redis 集群化 2.主从复制 1)主从复制主要的作用 2)快速体验 ①.第一步:本地启 ...
- Oracle Clusterware的命令集可以分为以下4种: 节点层:osnodes 网络层:oifcfg 集群层:crsctl, ocrcheck,ocrdump,ocrconfig 应用
Oracle Clusterware的命令集可以分为以下4种: 节点层:osnodes 网络层:oifcfg 集群层:crsctl, ocrcheck,ocrdump,ocrconfig 应用 ...
- 最新版Ambari2.75安装及HDP3.1.5集群搭建
最近打算使用ambari搭建集群,却突然发现Cloudera相关软件包括HDP.CDH都收费了,都需要有效的订阅,并且只能通过付费墙进行访问(官宣地址:关于Cloudera软件的访问获取).还好笔者千 ...
- Hadoop集群搭建(五:Hadoop HA集群模式的安装)
实验 目的 要求 目的: 1.Hadoop的高可用完全分布模式的安装和验证 要求: 完成Hadoop的高可用完全分布模式的安装: Hadoop的相关服务进程能够正常的启动: HDFS能够正常的使用: ...
最新文章
- document.readystate
- 源码解析:解析掌阅X2C 框架
- 直击微软MIX11 聚焦IE10、Silverlight5、WP7
- MAT类如何存储数据
- gvim配置及相关插件安装(过程详细,附图)
- 关于css3的自定义字体
- 【Flink】基于 Flink CEP 实时计算商品订单流失量
- Thread之四:java线程返回结果的方法
- JavaScript中值类型与引用类型
- 100套Java毕业设计和课程设计项目案例(包含项目源码)
- 161021_又是两个月还是总结
- 计算机的管理选项文件错误,如何修复Microsoft Teams错误文件在Windows PC中被锁定...
- vue 左侧菜单可拖动
- swfupload 无法加载_flash上传插件——swfupload 【原创】
- 自用frp服务器分享及客户端配置连接教程
- WSL和双系统Ubuntu的CPU性能差异
- ArcGIS二次开发基础教程(01):打开mxd文件,添加(.shp),(.lyr),(.mdb),(.tif;.img)等多种类型数据
- 2021年度总结 —— CSDN一位六万粉丝博主的成长之路
- 轻轻松松搞定分布式Token校验
- 自然语音处理(NLP)系列(五)——详解智能问答系统
热门文章
- powershell自动化操作AD域、Exchange邮箱系列(9)—导出AD内所有计算机到数据库
- wepe 装linux ubuntu,Ubuntu16.04下安装Wechat的实现方法
- 饿了么超时20分钟_饿了么:5分钟;美团:8分钟......消费者:???
- netty支持哪些协议_从零学习netty网络IO通讯开发框架
- samp服务器信息获取,手把手教你写一个简单的服务器
- 美国计算机科学奥林匹克竞赛试题,中学信息奥林匹克竞赛试题精选.doc
- webservice 安全性 对外_WebService安全性的几种实现方法【身份识别】
- Python解析access数据库(mdb文件或者accdb文件)
- 【机器学习-西瓜书】三、逻辑回归(LR);线性判别分析(LDA)
- B - Frogger(最短路之多条最短路径中最大权值的最小值)