hive insert overwrite directory 报错

hive insert overwrite directory 报错：

2017-06-07T16:56:11,766 ERROR [main]: exec.Task (:()) - Failed with exception Unable to move source hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000 to destination /tmp/t1
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move source hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000 to destination /tmp/t1at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:103)at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:254)at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.lang.reflect.Method.invoke(Method.java:497)at org.apache.hadoop.util.RunJar.run(RunJar.java:221)at org.apache.hadoop.util.RunJar.main(RunJar.java:136)Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename for src path: hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000/000000_0 to dest path:/tmp/t1/000000_0 returned falseat org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2942)at org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:118)at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:96)... 20 more
Caused by: java.io.IOException: rename for src path: hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000/000000_0 to dest path:/tmp/t1/000000_0 returned falseat org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2922)at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2911)at java.util.concurrent.FutureTask.run(FutureTask.java:266)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)at java.lang.Thread.run(Thread.java:745)

原因：
hive2.1.0bug，升级到hive2.1.1
报错原因是insert overwrite directory ，如果原来的directory已经有数据，mv会报错。详见HIVE-13997，地址https://issues.apache.org/jira/browse/HIVE-13997

bug修复思路：在mv结果数据到目标目录的之前，先删除目标目录的数据。这个策略与hive1.2.1版本不同。

打patch，https://issues.apache.org/jira/browse/HIVE-13997 下载HIVE-13997.2.patch，然后将path放到hive-2.1.0-src，执行patch -p1 < HIVE-13997.2.patch ; 然后重新编译ql模块，得到hive-exec-2.1.0.jar

hive insert overwrite directory 报错相关推荐

python打开文件时，找不到文件 FileNotFoundError: [Errno 2] No such file or directory报错
小白一枚,记录学习错误,共同进步 python打开文件时,找不到文件 FileNotFoundError: [Errno 2] No such file or directory报错 UnicodeD ...
Hive环境搭建启动报错
hive-site.xml文件内容: 1 <?xml version="1.0" encoding="UTF-8" standalone="no ...
Hive DML 语句操作报错
按照MySQL 方式创建的表, hive> desc tt; OK id int name string age int Time taken: 0.123 seconds, Fetched: ...
执行Hive的查询语句报错：java.lang.IllegalArgumentException: Does not contain a valid host:port authority: loca
好不容易把Hive装完了,结果一执行Hive的查询语句运行MapReduce程序立马报错... log详细信息如下: Job running in-process (local Hadoop) Had ...
mysql建表指定引擎_请教mysql建表指定data directory 报错
-- 如下: 语句 1... create table test(id int) engine=myisam data directory='/tmp' index directory='/tmp/a ...
Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient报错，问题排查...
背景最近在整合pyspark与hive,新安装spark-2.3.3以客户端的方式访问hive数据,运行方式使用spark on yarn,但是在配置spark读取hive数据的时候,这里直接把hi ...
Mybatis结合Oracle的foreach insert批量插入报错！
2019独角兽企业重金招聘Python工程师标准>>> 最近做一个批量导入的需求,将多条记录批量插入数据库中.解决思路:在程序中封装一个List集合对象,然后把该集合中的实体插入到数 ...
oracle匿名代码块执行insert,MyBatis+Oracle在执行insert时空值报错之从源码寻找解决办法...
mybatis-oracle-config.xml 复制代码 1 <?xml version="1.0" encoding="UTF-8"?> 2 ...
hive -e执行命令报错
执行hive -e命令报错 hive -e "alter table ods.article add partition (day='20201201') location 'hdfs:// ...

hive insert overwrite directory 报错

hive insert overwrite directory 报错相关推荐

最新文章

热门文章