beam+flink整合异常

信息: Remoting shut down.

十月 23, 2017 9:10:40 下午 org.apache.beam.runners.flink.FlinkRunner run

严重: Pipeline execution failed

org.apache.flink.client.program.ProgramInvocationException: The program execution failed: Couldn't retrieve the JobExecutionResult from the JobManager.

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:478)

at org.apache.flink.client.program.StandaloneClusterClient.submitJob(StandaloneClusterClient.java:105)

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:442)

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:429)

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:404)

at org.apache.flink.client.RemoteExecutor.executePlanWithJars(RemoteExecutor.java:211)

at org.apache.flink.client.RemoteExecutor.executePlan(RemoteExecutor.java:188)

at org.apache.flink.api.java.RemoteEnvironment.execute(RemoteEnvironment.java:172)

at org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:114)

at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:118)

at org.apache.beam.sdk.Pipeline.run(Pipeline.java:297)

at org.apache.beam.sdk.Pipeline.run(Pipeline.java:283)

at org.apache.beam.examples.WordCount.main(WordCount.java:184)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)

at java.lang.Thread.run(Thread.java:748)

Caused by: org.apache.flink.runtime.client.JobExecutionException: Couldn't retrieve the JobExecutionResult from the JobManager.

at org.apache.flink.runtime.client.JobClient.awaitJobResult(JobClient.java:309)

at org.apache.flink.runtime.client.JobClient.submitJobAndWait(JobClient.java:396)

at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:467)

... 18 more

Caused by: org.apache.flink.runtime.client.JobClientActorConnectionTimeoutException: Lost connection to the JobManager.

at org.apache.flink.runtime.client.JobClientActor.handleMessage(JobClientActor.java:219)

at org.apache.flink.runtime.akka.FlinkUntypedActor.handleLeaderSessionID(FlinkUntypedActor.java:101)

at org.apache.flink.runtime.akka.FlinkUntypedActor.onReceive(FlinkUntypedActor.java:68)

at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167)

at akka.actor.Actor$class.aroundReceive(Actor.scala:467)

at akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97)

at akka.actor.ActorCell.receiveMessage(ActorCell.scala:516)

at akka.actor.ActorCell.invoke(ActorCell.scala:487)

at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:238)

at akka.dispatch.Mailbox.run(Mailbox.scala:220)

at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)

at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)

at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)

at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

解决：beam和flink使用的通信是akka，而akka在通信时通常用主机名，而不用ip，因为我在flink的配置文件中使用的是IP地址，所以报上述错误，吧flink配置中的IP改成主机名，问题解决！

beam+flink整合异常相关推荐

Flink整合kafka的两阶段提交结论
文章目录 1.Flink+kafka是如何实现exactly-once语义的 2.WC案例的如何做chekcpoint 3.源码分析 4.kafkaConsumer在与Flink整合的思考 4.1 k ...
从零构建FLINK整合Drools动态规则实时运营系统（项目案例）第1篇（项目介绍篇）
1 整体架构前言项目介绍在线视频: https://www.bilibili.com/video/BV1zv41157yY 本案例是一个专注于flink动态规则计算的项目,核心技术组件涉及flin ...
Flink常见异常和错误信息小结
大数据技术与架构点击右侧关注,大数据开发领域最强公众号! 暴走大数据点击右侧关注,暴走大数据! Flink的常见异常众多,不可能面面俱到,所以想到哪儿写到哪儿,有漏掉的之后再补充. 部署和资源问题 ...
从零构建FLINK整合Drools动态规则实时运营系统（项目案例）第2篇（业务介绍篇）
项目简介本案例是一个专注于flink动态规则计算的项目,核心技术组件涉及flink.hbase.clickhouse.drools等项目可根据各类个性化需求进行二次开发后,直接用于实时运营,实时风 ...
flink整合java,Flink使用SideOutPut替换Split实现分流
基于apache flink的流处理实时模型 44元包邮 (需用券) 去购买 > 以前的数据分析项目(版本1.4.2),对从Kafka读取的原始数据流,调用split接口实现分流. 新项目决定 ...
Exceeded checkpoint tolerable failure threshould【记一次flink上线异常】
最近完成一个业务需求,flink程序上线之后,跑了不到一分钟就挂了,查看日志,报错如下: org.apache.flink.util.FlinkRuntimeException: Exceeded c ...
SSM整合-异常处理器及项目异常处理方案
目录一.异常处理器二.项目异常处理方案项目异常分类项目异常处理方案项目异常处理一.异常处理器程序开发过程中不可避免的会遇到异常现象出现异常现象的常见位置与常见诱因如下: ■ 框架内部抛 ...
Flink/Hbase 异常 - 4.Sink 背压100% 与 hbase.util.RetryCounter.sleepUntilNextRetry 异常分析与排查
一.引言 Flink 程序内有读取 hbase 的需求,近期任务启动后偶发 sink 端背压 100% 导致无数据写入下游且无明显 exception 报错,重启任务后有较大概率恢复服务,但也有可能继 ...
Flink相关异常解决
本文开头附:Flink 学习路线系列 ^ _ ^ 1.Flink Table API 中,错误信息如下: 错误信息: Exception in thread "main" org. ...

beam+flink整合异常

beam+flink整合异常相关推荐

最新文章

热门文章