2019独角兽企业重金招聘Python工程师标准>>>

https://community.hortonworks.com/questions/71589/after-upgrading-to-hdp-253-using-beeline-with-a-si.html

报错如下 参考链接如上 感觉是 orc的文件优化导致的这个错误 解决办法是

set hive.exec.orc.split.strategy=BI;

加上这么一个参数即可

Vertex failed, vertexName=Map 5, vertexId=vertex_1507250665075_0317_1_00, diagnostics=[Vertex vertex_1507250665075_0317_1_00 [Map 5] killed/failed due to:ROOT_INPUT_INIT_FAILURE, Vertex Input: erp_orders_union0124 initializer failed, vertex=vertex_1507250665075_0317_1_00 [Map 5], java.lang.RuntimeException: serious problem
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1172)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getSplits(OrcInputFormat.java:1199)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:307)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:409)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:155)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:273)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:266)
09-10-2017 08:52:55 CST append ERROR -  at java.security.AccessController.doPrivileged(Native Method)
09-10-2017 08:52:55 CST append ERROR -  at javax.security.auth.Subject.doAs(Subject.java:422)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:266)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
09-10-2017 08:52:55 CST append ERROR -  at java.util.concurrent.FutureTask.run(FutureTask.java:266)
09-10-2017 08:52:55 CST append ERROR -  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
09-10-2017 08:52:55 CST append ERROR -  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
09-10-2017 08:52:55 CST append ERROR -  at java.lang.Thread.run(Thread.java:745)
09-10-2017 08:52:55 CST append ERROR - Caused by: java.util.concurrent.ExecutionException: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1265061168-192.168.5.206-1490093965591:blk_1074553674_813300 file=/apps/hive/warehouse/erp2.db/erp_orders_union0124/000000_0
09-10-2017 08:52:55 CST append ERROR -  at java.util.concurrent.FutureTask.report(FutureTask.java:122)
09-10-2017 08:52:55 CST append ERROR -  at java.util.concurrent.FutureTask.get(FutureTask.java:192)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.generateSplitsInfo(OrcInputFormat.java:1167)
09-10-2017 08:52:55 CST append ERROR -  ... 15 more
09-10-2017 08:52:55 CST append ERROR - Caused by: org.apache.hadoop.hdfs.BlockMissingException: Could not obtain block: BP-1265061168-192.168.5.206-1490093965591:blk_1074553674_813300 file=/apps/hive/warehouse/erp2.db/erp_orders_union0124/000000_0
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hdfs.DFSInputStream.chooseDataNode(DFSInputStream.java:1004)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hdfs.DFSInputStream.blockSeekTo(DFSInputStream.java:648)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hdfs.DFSInputStream.readWithStrategy(DFSInputStream.java:898)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:955)
09-10-2017 08:52:55 CST append ERROR -  at java.io.DataInputStream.readFully(DataInputStream.java:195)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.extractMetaInfoFromFooter(ReaderImpl.java:370)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.ReaderImpl.(ReaderImpl.java:317)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcFile.createReader(OrcFile.java:238)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.populateAndCacheStripeDetails(OrcInputFormat.java:1057)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.callInternal(OrcInputFormat.java:969)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.access$1800(OrcInputFormat.java:810)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:960)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator$1.run(OrcInputFormat.java:957)
09-10-2017 08:52:55 CST append ERROR -  at java.security.AccessController.doPrivileged(Native Method)
09-10-2017 08:52:55 CST append ERROR -  at javax.security.auth.Subject.doAs(Subject.java:422)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:957)
09-10-2017 08:52:55 CST append ERROR -  at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat$SplitGenerator.call(OrcInputFormat.java:810)
09-10-2017 08:52:55 CST append ERROR -  ... 4 more
09-10-2017 08:52:55 CST append ERROR - ]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Reducer 4, vertexId=vertex_1507250665075_0317_1_03, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1507250665075_0317_1_03 [Reducer 4] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Reducer 3, vertexId=vertex_1507250665075_0317_1_02, diagnostics=[Vertex received Kill in INITED state., Vertex vertex_1507250665075_0317_1_02 [Reducer 3] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - Vertex killed, vertexName=Map 1, vertexId=vertex_1507250665075_0317_1_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:87, Vertex vertex_1507250665075_0317_1_01 [Map 1] killed/failed due to:OTHER_VERTEX_FAILURE]
09-10-2017 08:52:55 CST append ERROR - DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:3

加上之后不报这个错了 改成block missing

之后想起来 最近hdfs死过一回 应该就是那一回导致block missing

然后百度 hdfs fsck /

之后 hdfs fsck -delete / 删除丢失的块(因为我这儿的数据不怎么重要 所以直接删除)

再次执行 hive sql语句 ok 搞定

转载于:https://my.oschina.net/ilovetao/blog/1572768

hive日常报错之 serious problem serious problem,block 丢失相关推荐

  1. 日常问题——初始化Hive仓库报错com.google.common.base.Preconditions.checkArgument

    问题描述: 初始化Hive仓库报错Exception in thread "main" java.lang.NoSuchMethodError: com.google.common ...

  2. java小白日常报错

    java小白日常报错 CRM项目中使用逆向工程生成mappper.xml文件.pojo文件.和mapper文件时产生的错误* 作为一个java小白人,开始学习写博客,一个是为了为了记笔记,另一个是为了 ...

  3. Hive启动报错：java.lang.ClassNotFoundException: org.apache.tez.dag.api.TezConfiguration

    <!--Hive启动报错:Caused by: java.lang.ClassNotFoundException: org.apache.tez.dag.api.TezConfiguration ...

  4. Hive启动报错 java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang

    Hive启动报错 [lili@hadoop102 hive]$ bin/hive which: no hbase in (/usr/lib64/qt-3.3/bin:/usr/local/bin:/b ...

  5. Hive 启动报错java.net.URISyntaxException: Relative path in absolute URI 解决方法

    linux安装hive 遇到问题:Hive 启动报错java.net.URISyntaxException: Relative path in absolute URI 解决方法 方法记录 提示:这里 ...

  6. tez引擎跑hive 语句报错问题解决

    tez引擎跑hive 语句报错问题解决 现象: Container [pid=151320,containerID=***] is running beyond physical memory lim ...

  7. Hive查询报错：selectItem : ( ( exprexxxx ( ( ( KW_AS )......

    Hive查询报错: NoViableAltException(-1@[124:1: selectItem : ( ( expression ( ( ( KW_AS )? identifier ) | ...

  8. centos7 hive启动报错 没有到主机的路由，防火墙已关闭

    问题 hive启动报错,防火墙已关闭 at java.lang.reflect.Constructor.newInstance(Constructor.java:423)at org.datanucl ...

  9. ISE MAP报错： Unsupported programming for BSCAN block and JTAG_CHAIN attribute value 1的解决方法

    ISE MAP报错: Unsupported programming for BSCAN block and JTAG_CHAIN attribute value 1的解决方法 参考文章: (1)IS ...

最新文章

  1. SQL 中left join ,inner join,right join的区别
  2. 图像 主轴 相关知识
  3. 使用JAX-RS创建RESTful Web Service
  4. wxWidgets：wxValidator概述
  5. 数据结构实验之栈四：括号匹配
  6. 高中计算机教师招聘试题,中小学信息技术教师招聘考试真题及答案.doc
  7. mysql字符串语法_MySQL语法模板 函数:字符串
  8. 多核CPU上python多线程并行的一个假象(转)
  9. Linux 多播（组播）例程
  10. NYOJ4——ASCII码排序
  11. 第四季-专题17-LCD驱动程序设计
  12. 地铁关门提示音，可使用倒计时
  13. 【资源分享】Go语言并发之道 [美] 凯瑟琳（Katherine Cox-Buday）著 PDF 下载
  14. Java环境变量配置（详细操作）
  15. Android官方BottomNavigationView添加Badge(角标)，全部tab显示文字自动铺满
  16. Python 预测孩子身高
  17. ajax实现直播功能,jquery实现直播弹幕效果
  18. java唱哪首歌好听_非常非常好听，但是非常非常难找的歌
  19. 带你实现电商商品同款识别算法
  20. 什么是静态测试？什么是动态测试？

热门文章

  1. linux3.x 驱动程序,Arm Linux3.x的设备树 - 龙芯开源社区
  2. 富士施乐涉嫌走私重案
  3. 2020最新中高阶Android面试题总结 下（附解题思路）
  4. 回归Qt——写在Qt5.10发布之日
  5. 胜者举杯相庆,败者拼死相救
  6. Jupyter notebook 快捷键
  7. 汽车小型摄像机模块的全球与中国市场2022-2028年：技术、参与者、趋势、市场规模及占有率研究报告
  8. Python 各种库
  9. FTP虚拟用户及ssl加密配置
  10. mybatis标签set