在Hive中执行DDL之类的SQL语句时遇到的一个问题

作者:天齐

遇到的问题如下:

hive> create table ehr_base(id string);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.)

在解决此问题之前,先看一下Hive的安装过程。

一、Hive的安装

将Hive的安装包"apache-hive-1.2.1-bin.tar.gz"上传到linux服务器。

解压安装包,将解压后的目录重命名为hive,进入到hive/conf目录,创建hive-site.xml文件:

[root@hadoop01 apps]# ls apache-hive-1.2.1-bin.tar.gz
apache-hive-1.2.1-bin.tar.gz
[root@hadoop01 apps]# tar -zxvf apache-hive-1.2.1-bin.tar.gz
[root@hadoop01 apps]# mv apache-hive-1.2.1-bin hive
[root@hadoop01 apps]# cd hive/
[root@hadoop01 hive]# ll
total 476
drwxr-xr-x. 3 root root   4096 Jan 31 22:07 bin
drwxr-xr-x. 2 root root   4096 Jan 31 23:33 conf
drwxr-xr-x. 4 root root   4096 Jan 31 22:07 examples
drwxr-xr-x. 7 root root   4096 Jan 31 22:07 hcatalog
drwxr-xr-x. 4 root root   4096 Jan 31 23:39 lib
-rw-rw-r--. 1 root root  24754 Apr 29  2015 LICENSE
-rw-rw-r--. 1 root root    397 Jun 19  2015 NOTICE
-rw-rw-r--. 1 root root   4366 Jun 19  2015 README.txt
-rw-rw-r--. 1 root root 421129 Jun 19  2015 RELEASE_NOTES.txt
drwxr-xr-x. 3 root root   4096 Jan 31 22:07 scripts
[root@hadoop01 apps]# cd hive/
[root@hadoop01 hive]# ll
total 476
drwxr-xr-x. 3 root root   4096 Jan 31 22:07 bin
drwxr-xr-x. 2 root root   4096 Jan 31 23:33 conf
drwxr-xr-x. 4 root root   4096 Jan 31 22:07 examples
drwxr-xr-x. 7 root root   4096 Jan 31 22:07 hcatalog
drwxr-xr-x. 4 root root   4096 Jan 31 23:39 lib
-rw-rw-r--. 1 root root  24754 Apr 29  2015 LICENSE
-rw-rw-r--. 1 root root    397 Jun 19  2015 NOTICE
-rw-rw-r--. 1 root root   4366 Jun 19  2015 README.txt
-rw-rw-r--. 1 root root 421129 Jun 19  2015 RELEASE_NOTES.txt
drwxr-xr-x. 3 root root   4096 Jan 31 22:07 scripts
[root@hadoop01 hive]# cd conf/
[root@hadoop01 conf]# vi hive-site.xml
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--Licensed to the Apache Software Foundation (ASF) under one or morecontributor license agreements.  See the NOTICE file distributed withthis work for additional information regarding copyright ownership.The ASF licenses this file to You under the Apache License, Version 2.0(the "License"); you may not use this file except in compliance withthe License.  You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
-->
<configuration><property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://hadoop03:3306/hive?createDatabaseIfNotExist=true</value><description>JDBC connect string for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value><description>Driver class name for a JDBC metastore</description></property><property><name>javax.jdo.option.ConnectionUserName</name><value>hive</value><description>username to use against metastore database</description></property><property><name>javax.jdo.option.ConnectionPassword</name><value>123456</value><description>password to use against metastore database</description></property>
</configuration>

二、拷贝MySQL驱动jar包到hive

[root@hadoop01 lib]# pwd
/root/apps/hive/lib
[root@hadoop01 lib]# ls mysql-connector-java-5.1.39.jar
mysql-connector-java-5.1.39.jar
[root@hadoop01 lib]# 

三、在MySQL中创建Hive用户并授权

[root@hadoop01 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.6.35 MySQL Community Server (GPL)Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> CREATE USER 'hive'@'%' IDENTIFIED BY '123456';
Query OK, 0 rows affected (0.12 sec)mysql> GRANT all privileges ON hive.* TO 'hive'@'%';
Query OK, 0 rows affected (0.00 sec)mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)mysql> exit;
Bye
[root@hadoop01 ~]# mysql -uhive -p
Enter password:
ERROR 1045 (28000): Access denied for user 'hive'@'localhost' (using password: YES)
[root@hadoop01 ~]# mysql -uhive -p123456
Warning: Using a password on the command line interface can be insecure.
ERROR 1045 (28000): Access denied for user 'hive'@'localhost' (using password: YES)
[root@hadoop01 ~]# mysql -u hive -p 123456
Enter password:
ERROR 1045 (28000): Access denied for user 'hive'@'localhost' (using password: YES)
[root@hadoop01 ~]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.6.35 MySQL Community Server (GPL)Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> grant all on hive.* to hive@' gc.localdomain' identified by '123456';
Query OK, 0 rows affected (0.00 sec)mysql> grant all on hive.* to hive@' localhost' identified by '123456';
Query OK, 0 rows affected (0.00 sec)mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)mysql> exit;
Bye
[root@hadoop01 ~]# mysql -uhive -p123456
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 7
Server version: 5.6.35 MySQL Community Server (GPL)Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
+--------------------+
1 row in set (0.06 sec)mysql> 

四、启动Hive并建库建表

在创建表的时候报如下错误(其他类似DDL的sql语句都会报此错误):

[root@hadoop01 hive]# pwd
/root/apps/hive
[root@hadoop01 hive]# bin/hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/apps/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/apps/spark/lib/spark-assembly-1.6.2-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/apps/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/apps/spark/lib/spark-assembly-1.6.2-hadoop2.6.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]Logging initialized using configuration in jar:file:/root/apps/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive> show databases;
OK
default
Time taken: 3.135 seconds, Fetched: 1 row(s)
hive> create database db1;
OK
Time taken: 0.461 seconds
hive> show databases;
OK
db1
default
Time taken: 0.105 seconds, Fetched: 2 row(s)
hive> use db1;
OK
Time taken: 0.076 seconds
hive> create table tb_1(id string);
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException
(message:For direct MetaStore DB connections, we don't support retries at the client level.)

五、解决办法

经过网络查询,说是MySQL中的hive数据库字符编码有问题,需要连接mysql服务器,并执行SQL语句修改hive数据库的字符编码:

[root@hadoop01 soft]# mysql -uhive -p123456
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 27
Server version: 5.6.35 MySQL Community Server (GPL)Copyright (c) 2000, 2016, Oracle and/or its affiliates. All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.mysql> alter database hive character set latin1;
Query OK, 1 row affected (0.00 sec)mysql>

再次建表,问题解决:

hive> create table tb_1(id string);
OK
Time taken: 3.018 seconds
hive> show tables;
OK
tb_1
Time taken: 0.13 seconds, Fetched: 1 row(s)
hive> show tables;
OK
tb_1
Time taken: 0.13 seconds, Fetched: 1 row(s)
hive> insert into tb_1 values('001');
Query ID = root_20170131232220_8dd5fa31-97b3-4688-9110-a59840064045
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks is set to 0 since there is no reduce operator
Starting Job = job_1485929365702_0001, Tracking URL = http://hadoop01:8088/proxy/application_1485929365702_0001/
Kill Command = /root/apps/hadoop/bin/hadoop job  -kill job_1485929365702_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2017-01-31 23:23:32,336 Stage-1 map = 0%,  reduce = 0%
2017-01-31 23:24:03,840 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 4.44 sec
MapReduce Total cumulative CPU time: 4 seconds 440 msec
Ended Job = job_1485929365702_0001
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to: hdfs://ns1/user/hive/warehouse/db1.db/tb_1/.hive-staging_hive_2017-01-31_23-22-20_362_3014216082832716531-1/-ext-10000
Loading data to table db1.tb_1
[Warning] could not update stats.
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1   Cumulative CPU: 4.44 sec   HDFS Read: 3133 HDFS Write: 72 SUCCESS
Total MapReduce CPU Time Spent: 4 seconds 440 msec
OK
Time taken: 130.905 seconds
hive> select * from tb_1;
OK
001
Time taken: 0.377 seconds, Fetched: 1 row(s)
hive> 

补充:但是,也有说是mysql的驱动jar包版本有问题,我也更换了几次驱动jar包的版本,但是貌似都不起作用。

[文章结束]

在Hive中执行DDL之类的SQL语句时遇到的一个问题相关推荐

  1. 如何查询mysql中执行效率低的sql语句

    一些小技巧 1. 如何查出效率低的语句? 在MySQL下,在启动参数中设置 --log-slow-queries=[文件名],就可以在指定的日志文件中记录执行时间超过long_query_time(缺 ...

  2. mysql执行效率低_如何查询mysql中执行效率低的sql语句

    展开全部 一.MySQL数据库有几个配置选项可以32313133353236313431303231363533e59b9ee7ad9431333337373563帮助我们及时捕获低效SQL语句 1, ...

  3. springboot+Mybatis+MySql 一个update标签中执行多条update sql语句

    MySql默认是不支持这种骚操作的,但是并不代表不能实现,只需要在jdbc的配置文件中稍做配置: 配置文件:jdbc.properties driver=com.mysql.jdbc.Driver u ...

  4. Mybatis 一个update标签中执行多条update sql语句

    本质上Mysql是不支持这种骚操作的 但是不代表并不能实现,只需要在jdbc的url链接库地址上添加&allowMultiQueries=true即可 driver=com.mysql.jdb ...

  5. Bash中执行存储过程或普通的SQL命令

    演示: 在bash中执行一个存储过程和一个普通的SQL语句 0. 前提条件: 数据库:Oracle10g 表:oracle自带的emp表,默认有数据如下: 1. 在bash中执行oracle存储过程 ...

  6. sql定位过程报错_如何得到plsql中执行时报错的SQL的位置行号

    如何得到 PL/SQL 中执行时报错的 SQL 的位 置行号 一.引言 在进行 PL/SQL 开发的时候经常会写以下代码, declare n1 varchar(10); n2 varchar(10) ...

  7. IDEA中安装MyBatis Log Plugin插件完整显示执行的mybatis的sql语句

    场景 在IDEA中mybatis执行的sql语句会将参数输出为? 这样的sql语句时不完整的,不利于后续的sql语句排错等. MyBatis Log Plugin 这款插件是直接将Mybatis执行的 ...

  8. 预算执行审计计算机审计方法,预算执行审计中计算机审计思路和SQL语句运用.doc...

    预算执行审计中计算机审计思路和SQL语句运用 预算执行审计中计算机审计思路和SQL语句运用 [摘要] 面对庞大的业务数据量,计算机辅助审计的开展解决了手工审计无法做到的审查功能,财政预算执行的审计领域 ...

  9. mysql查询低效语句_MySQL数据库中查找执行从命慢的SQL语句

    MySQL数据库中查找执行从命慢的SQL语句 (2011-09-15 08:21:35) 标签: 杂谈 去历:赛迪网 做者:Alizze 启动Mysql时减参数--log-slow-queries去挤 ...

最新文章

  1. 最近做了一个博客 玩玩而已 运城搜搜 www.lenovoyh.com
  2. Cell Reports:黄梓芮等揭示大脑皮层中通往意识的“大门”
  3. abaqus推荐用哪一版本的_2020年双十一哪一款立式空调/柜式空调值得推荐?【立式空调推荐/柜式空调推荐】·精选...
  4. 人脸属性迁移学习笔记
  5. Winfrom实用代码项目
  6. Egret 生成 自带EUI 的微信小游戏 踩坑!
  7. Python中内置函数的介绍
  8. ASP.NET Core开发Docker部署
  9. THAAD反导必将部署,各方已接受事实
  10. maven的pom详解
  11. 传奇服务器修改二级密码,找回传奇玩家账号和二级密码,修改密码和密保
  12. ubuntu+python2.7+tensorflow-gpu安装
  13. 利用 Python 爬取58同城品牌公寓(附带租房指南)
  14. 你有什么经验一定要分享给初入职场的新人?
  15. 模拟电路单片机硬件比例运算放大器LM358通过PROTUES演示
  16. win7系统mscomctl.ocx不能注册
  17. C语言计算5+55+555+5555+55555
  18. 多元函数微分学——全微分和微分中高阶无穷小的联系
  19. mac使用php无法写入文件,mac移动硬盘无法写入怎么办
  20. 福建计算机信息技术大学排名,2017福建所有的大学以及排名情况

热门文章

  1. SunDay天气——开放源代码
  2. Linux初级运维(七)——bash脚本编程(常见测试)
  3. Hadoop版本:CDH, HDP, MapR
  4. Java 二次MD5 32位小写加密算法与php页面加密结果相同
  5. Android移动端音视频的快速开发教程(十)
  6. Ubuntu 安装flash
  7. 技术感悟---主动学习
  8. IE 无法点击文本框或输入文字
  9. 从键盘输入4个学生的有关数据,然后把它们保存到磁盘文件中,最后从磁盘文件中读取数据输出到屏幕...
  10. 微信小程序绑定数据以及自定义指令