Flume sink=avro rpc connection error
要求
conf 文件
a1.sources=r1
a1.sinks=k1
a1.channels=c1a1.sources.r1.type=avro
a1.sources.r1.bind=master
a1.sources.r1.port=9999a1.sinks.k1.type=avro
a1.sinks.k1.hostname=slave1
a1.sinks.k1.port=7777a1.channels.c1.type=memory
a1.channels.c1.capacity=1000
a1.channels.c1.transactionCapacity=100a1.sources.r1.channels=c1
a1.sinks.k1.channel=c1
第二个conf
a1.sources=r1
a1.sinks=k1
a1.channels=c1a1.sources.r1.type=avro
a1.sources.r1.bind=slave1
a1.sources.r1.port=7777a1.sinks.k1.type=loggera1.channels.c1.type=memory
a1.channels.c1.capacity=1000
a1.channels.c1.transactionCapacity=100a1.sources.r1.channels=c1
a1.sinks.k1.channel=c1
当运行第第一个conf
出现错误
org.apache.flume.FlumeException: NettyAvroRpcClient { host: slave1, port: 8888 }: RPC connection errorat org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:181)at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:120)at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:90)at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:210)at org.apache.flume.sink.AbstractRpcSink.start(AbstractRpcSink.java:290)at org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:45)at org.apache.flume.SinkRunner.start(SinkRunner.java:79)at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:249)at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Error connecting to slave1/192.168.111.199:8888at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:169)... 16 more
Caused by: java.net.ConnectException: Connection refused: slave1/192.168.111.199:8888at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)... 3 more
17/10/06 21:52:54 INFO sink.AbstractRpcSink: Rpc sink s1 started.
17/10/06 21:52:54 INFO sink.AbstractRpcSink: Rpc sink s1: Building RpcClient with hostname: slave1, port: 8888
17/10/06 21:52:54 INFO sink.AvroSink: Attempting to create Avro Rpc client.
17/10/06 21:52:54 WARN api.NettyAvroRpcClient: Using default maxIOWorkers
17/10/06 21:52:54 ERROR flume.SinkRunner: Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: Failed to send eventsat org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:389)at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:67)at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:145)at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.flume.FlumeException: NettyAvroRpcClient { host: slave1, port: 8888 }: RPC connection errorat org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:181)at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:120)at org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:638)at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:90)at org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:127)at org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:210)at org.apache.flume.sink.AbstractRpcSink.verifyConnection(AbstractRpcSink.java:270)at org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:346)... 3 more
Caused by: java.io.IOException: Error connecting to slave1/192.168.111.199:8888at org.apache.avro.ipc.NettyTransceiver.getChannel(NettyTransceiver.java:261)at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:203)at org.apache.avro.ipc.NettyTransceiver.<init>(NettyTransceiver.java:152)at org.apache.flume.api.NettyAvroRpcClient.connect(NettyAvroRpcClient.java:169)... 10 more
Caused by: java.net.ConnectException: Connection refused: slave1/192.168.111.199:8888at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:318)at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)... 1 more
原因是先后顺序不对 得先打开第二个conf这样才可以保证sink的avro输出有接收方
Flume sink=avro rpc connection error相关推荐
- Flume的Avro Sink和Avro Source研究之一: Avro Source
问题 : Avro Source提供了怎么样RPC服务,是怎么提供的? 问题 1.1 Flume Source是如何启动一个Netty Server来提供RPC服务. 由GitHub上avro-rpc ...
- Flume整合SparkStreaming出现UnsupportedClassVersionError: org/apache/spark/streaming/flume/sink/SparkFlum
Flume整合SparkStreaming使用Poll方式拉取数据出现 UnsupportedClassVersionError: org/apache/spark/streaming/flume/s ...
- Flume Sink
一 HDFS Sink 将event写入HDFS. 1.1 RequiredProperties channel:从哪一个channelpull数据 type:hdfs hdfs.path:HDFS ...
- flume sink hdfs异常
1.异常消息如下: 016-08-26 14:19:17,704 (hdfs-sink1-call-runner-2) [ERROR - org.apache.flume.sink.hdfs.Abst ...
- AndroidStudio更新时报错:Connection Error,Temp directory inside installation
场景 在将Android Studio的 .AndroidStudio目录修改为别的目录后,打开AS,提示更新,点击更新后提示: Connection Error,Temp directory ins ...
- JetBrains——账户登录错误(JetBrains Account Error:JetBrains Account connection error: www.jetbrains.com)解决方案
问题描述 在用申请的学生免费JetBrains Account登录pycharm时 弹出报错框 JetBrains Account connection error: www.jetbrains.co ...
- Adb connection Error:远程主机强迫关闭了一个现有的连接
[2013-07-19 09:32:07 - DeviceMonitor] Adb connection Error:远程主机强迫关闭了一个现有的连接. [2013-07-19 09:32:08 - ...
- 大数据学习笔记53:Flume Sink Processors(Flume接收器处理器)
文章目录 一.Flume Sink Processors用户指南 二.Default Sink Processor 三.FailOver Sink Processor 四.Load Balancing ...
- Android Studio 完美解决 “Android SDK Manager 无法更新“、 ”connection error” 的问题...
一.Android SDK Manager 无法更新 1. 打开SDK Mannger, 并选中启动单独的SDK Mannger. 2. 打开Android SDK Mannger并设置Option ...
最新文章
- 最小树形图及其生产方法
- Mybatis-Plus 之 人生中荒废的一下午
- 小学计算机课的活动设计方案,小学信息技术兴趣小组活动策划书三篇
- java异常_Java的异常体系
- 【渝粤题库】广东开放大学 婚姻家庭法 形成性考核
- pb90代码如何连接sql2008r2_RabbitMQ各种交换机机制,代码实践篇
- php output详解,【PHP】Output Control 扩展详细解读
- P3372 【模板】线段树 1 区间查询与区间修改
- php学习第一讲----php是什么?
- UIPickerView的使用
- hdu4757(可持久化 Trie )
- 学习PetShop3.0(9)工厂的资料
- 系统架构设计-计算机组成与体系结构
- Mininet-wifi安装和简单使用
- 关于百度网盘下载过慢的解决方法
- 卡通的平板卧推男孩动画特效
- python做毕设有哪些好的题目?
- Spring-SecondDay
- 英特尔遭遇「灾难级」财报!裁员、砍业务、董事会主席辞职,甚至给不出2023业绩预期
- 无法远程服务器是什么原因?该怎么处理?