camus执行任务,偶发性异常
异常信息:
*********************************
MapReduce异常信息
*********************************
2015-07-20 14:16:01,045 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert; Ignoring.
2015-07-20 14:16:01,049 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-07-20 14:16:01,050 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf; Ignoring.
2015-07-20 14:16:01,053 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class; Ignoring.
2015-07-20 14:16:01,057 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring.
2015-07-20 14:16:01,081 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-07-20 14:16:01,358 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2015-07-20 14:16:01,463 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-07-20 14:16:01,463 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started
2015-07-20 14:16:01,479 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens:
2015-07-20 14:16:01,479 INFO [main] org.apache.hadoop.mapred.YarnChild: Kind: mapreduce.job, Service: job_1435737529069_0025, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@63a6d923)
2015-07-20 14:16:01,631 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.
2015-07-20 14:16:02,065 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /mnt/sdb1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdc1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdd1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sde1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdf1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025,/mnt/sdg1/yarn/nmx/usercache/root/appcache/application_1435737529069_0025
2015-07-20 14:16:02,271 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.require.client.cert; Ignoring.
2015-07-20 14:16:02,274 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring.
2015-07-20 14:16:02,275 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.client.conf; Ignoring.
2015-07-20 14:16:02,276 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.keystores.factory.class; Ignoring.
2015-07-20 14:16:02,279 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: hadoop.ssl.server.conf; Ignoring.
2015-07-20 14:16:02,295 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring.
2015-07-20 14:16:02,668 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
2015-07-20 14:16:03,198 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]
2015-07-20 14:16:03,454 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: com.linkedin.camus.etl.kafka.mapred.EtlSplit@d9f20e5
2015-07-20 14:16:03,582 INFO [main] org.apache.hadoop.io.compress.zlib.ZlibFactory: Successfully loaded & initialized native-zlib library
2015-07-20 14:16:03,584 INFO [main] org.apache.hadoop.io.compress.CodecPool: Got brand-new compressor [.deflate]
2015-07-20 14:16:03,595 INFO [main] com.linkedin.camus.etl.kafka.mapred.EtlRecordReader:
topic:hive-xm-syslog5 partition:0 beginOffset:80 estimatedLastOffset:120
2015-07-20 14:16:03,603 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: bufferSize=10485760
2015-07-20 14:16:03,603 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: timeout=30000
2015-07-20 14:16:04,029 INFO [main] com.linkedin.camus.etl.kafka.common.KafkaReader: Connected to leader tcp://m105:9092 beginning reading at offset 80 latest offset=120
2015-07-20 14:16:05,657 INFO [main] org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2015-07-20 14:16:05,702 INFO [main] org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize called
2015-07-20 14:16:06,018 INFO [main] DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored
2015-07-20 14:16:06,019 INFO [main] DataNucleus.Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored
2015-07-20 14:16:06,686 ERROR [BoneCP-pool-watch-thread] com.jolbox.bonecp.PoolWatchThread: Error in trying to obtain a connection. Retrying in 7000ms
java.sql.SQLException: A read-only user or a user in a read-only datab
ase is not permitted to disable read-only mode on a connection.
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
at org.apache.derby.impl.jdbc.EmbedConnection.setReadOnly(Unknown Source)
at com.jolbox.bonecp.ConnectionHandle.setReadOnly(ConnectionHandle.java:1324)
at com.jolbox.bonecp.ConnectionHandle.<init>(ConnectionHandle.java:262)
at com.jolbox.bonecp.PoolWatchThread.fillConnections(PoolWatchThread.java:115)
at com.jolbox.bonecp.PoolWatchThread.run(PoolWatchThread.java:82)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.sql.SQLException: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
... 13 more
Caused by: ERROR 25505: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection.
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericAuthorizer.setReadOnlyConnection(Unknown Source)
at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.setReadOnly(Unknown Source)
... 8 more
原因及方案解决:
m102节点找不到对应的Hive表元数据地址,造成程序偶尔运行异常,在hive-site.xml中配置Hive元数据信息
<property>
<name>hive.metastore.uris</name>
<value>thrift://m103:9083,thrift://m105:9083</value>
</property>
转载于:https://my.oschina.net/u/1474514/blog/484253
camus执行任务,偶发性异常相关推荐
- ThreadPoolExecutor执行任务,异常日志缺失问题
ThreadPoolExecutor执行任务,异常日志缺失问题 参考文章: (1)ThreadPoolExecutor执行任务,异常日志缺失问题 (2)https://www.cnblogs.com/ ...
- 基于ABP的应用执行Update-Database出现异常的解决办法
基于ABP的应用执行Update-Database出现异常的解决办法 参考文章: (1)基于ABP的应用执行Update-Database出现异常的解决办法 (2)https://www.cnblog ...
- Ubuntu 执行 bash 脚本异常
Ubuntu 执行 bash 脚本异常 文章目录 Ubuntu 执行 bash 脚本异常 执行报错 解决办法 参考 执行报错 [dev@my-ubuntu ~/packages/base_server ...
- android timer后函数继续执行_Android内存异常机制(用户空间)_JE
常见的Android稳定性异常,有内核异常和Android层异常.内核异常也就是常说的"kernel panic",简称KE异常:Android层异常又分为java层crash和N ...
- java 代码段 执行超时 抛异常_Java 并发队列详解TransferQueue、BlockingQueue、BlockingDeque
阻塞队列 BlockingQueue java.util.concurrent 包里的 BlockingQueue 接口表示一个线程安放入和提取实例的队列. BlockingQueue 用法 Bloc ...
- 解决 List 执行 remove 时报异常 java.lang.UnsupportedOperationException
前些天发现了一个巨牛的人工智能学习网站,通俗易懂,风趣幽默,忍不住分享一下给大家.点击跳转到教程. 一.情况描述 报错如题: java.lang.UnsupportedOperationExcepti ...
- 通达OA 工作流执行出现的异常现象处理(图文)
一个朋友用的工作流,突然说这里面多了很多人不应该看到这个流程的,就是在查询工作时,最上面多了一个"查阅信息"项,然后最下面多了很多人查询的记录.这个项目是从哪里来的呢?普通的流程倒 ...
- java实训 :异常(try-catch执行顺序与自定义异常)
关键字: try:执行可能产生异常的代码 catch:捕获异常 finally:无论是否发生异常代码总能执行 throws:声明方法可能要抛出的各种异常 throw:手动抛出自定义异常 用 try-c ...
- java怎么碰到异常跳过继续执行_Java异常处理很难吗?BAT大厂的架构师是怎么处理Java异常的?...
欢迎关注专栏<Java架构筑基>--专注于Java技术的研究与分享! Java架构筑基zhuanlan.zhihu.com Java架构筑基--专注于Java技术的研究与分享! 后续文章 ...
最新文章
- AMD yes!拿下Meta后发布新芯片,FP64性能是A100 4.9倍,面向高性能计算和机器学习...
- 2008-07-29
- 【数值分析】迭代法解方程:牛顿迭代法、Jacobi迭代法
- 国外值得关注的网站系列之二-社交化推荐网站GetGlue
- Android下Cocos2d创建HelloWorld工程
- 虚拟货币公有链项目集体爆发,AE超过历史最高点
- AI换脸APP“ZAO”刷屏并一夜爆火,它能红多久?
- 【实例解析】某集团BI决策系统建设方案分享
- oracle支持ipv6,准备现有的网络以支持 IPv6
- Educational Codeforces Round 52: E. Side Transmutations(burnside引理)
- MySQL 性能优化神器 Explain 使用分析
- linux系统多大分区,linux系统中fdisk最大能认到多大分区
- Java 经纬度转UTM投影
- mybatis文档地址
- 如何实现混合线性模型?
- 《从0到1》读书笔记第10章“打造帮派文化”第1记: PayPal黑帮花名册
- 胡侃学习计算机--理论之外-From南大小百合
- 【MOOC】计算机网络与通信技术笔记(北交)(3)数据链路层
- 股指指数平稳性检验和随机性检验
- chrome去除蓝色边框和黄色背景色
热门文章
- img图片加载失败时的处理
- js 截取某个字符前面或者后面的字符串
- 计算机科技文献中cam,模具加工论文,关于CAD/CAM在高精密模具设计制造中的应用相关参考文献资料-免费论文范文...
- Ubuntu1804安装CUDA与CUDNN
- 远期、期货和互换(三)
- 基于matlab数字交换网络的仿真,基于MATLAB数字交换网络的仿真
- 高颜值可视化设计UNIAPP源码生成器
- PP01工艺路线批量导入_SAP刘梦_新浪博客
- CMWAP和CMNET 的区别与适用范围
- 学计算机的演员,南开大学计算机系到演员 张桐回顾“不安分”的青春