hive报错整理之Malformed ORC file 、Invalid postscript.
Caused by: java.io.IOException: Malformed ORC file
将本地文件的数据加载到hive的ORC格式表时，出现以下报错：
Diagnostic Messages for this Task:
Error: java.io.IOException: java.lang.reflect.InvocationTargetExceptionat org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:295)at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.<init>(HadoopShimsSecure.java:242)at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileInputFormatShim.getRecordReader(HadoopShimsSecure.java:356)at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:591)at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.<init>(MapTask.java:168)at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:409)at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)at java.security.AccessController.doPrivileged(Native Method)at javax.security.auth.Subject.doAs(Subject.java:415)at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.reflect.InvocationTargetExceptionat sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)at java.lang.reflect.Constructor.newInstance(Constructor.java:526)at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.initNextRecordReader(HadoopShimsSecure.java:281)... 11 more
Caused by: java.io.IOException: Malformed ORC file hdfs://bigdata01.ibeifeng.com:8020/user/hive/warehouse/track_logs.db/track_log/date=20160323/20160323. Invalid postscript.at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.ensureOrcFooter(ReaderImpl.java:226)at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.extractMetaInfoFromFooter(ReaderImpl.java:336)at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.<init>(ReaderImpl.java:292)at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:197)at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:997)at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.<init>(CombineHiveRecordReader.java:65)... 16 moreFAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 Reduce: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec原因：
ORC格式是列式存储的表，不能直接从本地文件导入数据，只有当数据源表也是ORC格式存储时，才可以直接加载，否则会出现上述报错。
解决办法：
要么将数据源表改为以ORC格式存储的表，要么新建一个以textfile格式的临时表先将源文件数据加载到该表，然后在从textfile表中insert数据到ORC目标表中。
hive报错整理之Malformed ORC file 、Invalid postscript.相关推荐
- 【Hive】Hive查询报错 Malformed ORC file、Invalid postscript、serious problem
报错1: [2020-09-01 20:27:48] hive:Failed with exception java.io.IOException:java.lang.RuntimeException ...
- Presto读Hive报错： Malformed ORC file. Cannot read SQL type ‘tinyint‘ from ORC stream...
向社区提了个Issue: Malformed ORC file. Cannot read SQL type 'boolean' from ORC stream '.illegal' of type I ...
- hive报错Could not get block locations. Source file “/user/hive/warehouse/xxx
hive报错 Could not get block locations. Source file "/user/hive/warehouse/xxx... 以及 Caused by: or ...
- hive报错(org.apache.hadoop.hdfs.server.namenode.SafeModeException): Cannot create directory/tmp/hive/.
报错场景: 使用shell脚本进行对hive的数据查询导入导致出错,先是hive执行时间较长,后面hive报错. 问题描述: 使用jps查询进程,发现有三个hive进程,三个RunJar,RunJar ...
- ubuntu 找不到mysql.h_ubuntu安装了mysql 但是编译报错 mysql.h: No such file or directory
在Ubuntu体系中,已经安装了mysql,即应用sudo apt-get install mysql-server mysql-client 但是用C编译mysql数据库时,报错 fatal err ...
- Android Studio 的一些 常用操作和 编译报错 整理（持续更新中...）
Android Studio 的一些 常用操作和 编译报错 整理(持续更新中...) 目录 Android Studio 的一些 常用操作和 编译报错 整理(持续更新中...) 一.Android s ...
- Eslint报错整理与解决方法（持续整理）
Eslint报错整理与解决方法(持续整理) 参考文章: (1)Eslint报错整理与解决方法(持续整理) (2)https://www.cnblogs.com/lunlunshiwo/p/859600 ...
- #Hive报错 WritableStringObjectInspector cannot be cast to org.apache.hadoop.hive.serde2.objectinspect
#Hive报错 FAILED: ClassCastException org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableS ...
- 启动hive报错：java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang
报错详情: b/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf ...
最新文章
- nginx系列：nginx反向缓存代理详解
- Cygwin-添加到右键菜单脚本--一键安装、卸载
- heartbeat+drbd+mysql:实现最廉价的高可用组合
- c++判断一个数字是否为buzz number的算法(附完整源码)
- 操作系统源代码_国产操作系统“之光”？Windows XP绝密源代码泄露，BT种子已在网上疯传...
- TensorFlow的可训练变量和自动求导机制
- LeetCode 82. 删除排序链表中的重复元素 II（链表）
- redhat linux 9.0 拷贝u盘的文件,肿么用U盘安装Linux,安装的是red hat 9.0…用Ubuntu很方便,redhat可以吗?...
- 命令行远程链接MySQL
- 【转】到底什么时候应该用MQ
- Visual Sourcesafe Internet使用备忘
- 在linux服务器上安装jdk
- 红魔5S游戏手机与努比亚watch闪耀ChinaJoy
- 【Win7开启DMA】
- 如何关闭“数据执行保护”？
- meta标签详解（name、http-equiv、scheme、charset、各浏览器常用meta标签、常见移动端meta标签）、viewport详解
- 异步加载loading
- B树与B+树简明扼要的区别
- 自力更生给笔记本电脑换固态并重装系统
- 简易画图工具(Python)