HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))

HDFS频繁报如下错误：

2022-02-28 00:15:39,115 WARN datanode.DataNode (DataXceiverServer.java:run(168)) - hadoop-04:50010:DataXceiverServer:
java.io.IOException: Xceiver count 202 exceeds the limit of concurrent xcievers: 200
at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:150)
at java.lang.Thread.run(Thread.java:748)
2022-02-28 00:15:39,217 WARN datanode.DataNode (DataXceiverServer.java:run(168)) - hadoop-04:50010:DataXceiverServer:
java.io.IOException: Xceiver count 202 exceeds the limit of concurrent xcievers: 200
at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:150)
at java.lang.Thread.run(Thread.java:748)
2022-02-28 00:15:39,259 INFO DataNode.clienttrace (DataXceiver.java:requestShortCircuitFds(425)) - src: 127.0.0.1, dest: 127.0.0.1, op: REQUEST_SHORT_CIRCUIT_FDS, blockid: 1109394163, srvID: 2a966b4d-4941-4e55-9961-1a7f46a26b80, success: true
2022-02-28 00:15:39,294 INFO DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1533)) - src: /172.25.4.65:18826, dest: /172.25.4.65:50010, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-44093795_1, offset: 0, srvID: 2a966b4d-4941-4e55-9961-1a7f46a26b80, blockid: BP-734843239-172.25.4.62-1610084212828:blk_1109394784_35656232, duration(ns): 333214502
2022-02-28 00:15:39,294 INFO datanode.DataNode (BlockReceiver.java:run(1506)) - PacketResponder: BP-734843239-172.25.4.62-1610084212828:blk_1109394784_35656232, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=2:[172.25.4.64:50010, 172.25.4.63:50010] terminating
2022-02-28 00:15:39,297 INFO datanode.DataNode (DataXceiver.java:writeBlock(738)) - Receiving BP-734843239-172.25.4.62-1610084212828:blk_1109394792_35656240 src: /172.25.4.65:18832 dest: /172.25.4.65:50010

解决方法：

Hadoop HDFS DataNode 在任何时候都可以提供文件数量的上限。上限由 dfs.datanode.max.transfer.threads属性控制。修改hdfs-site.xml配置为至少4096(HDP集群默认200)。

HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))相关推荐

java项目导入包报错_转！java web项目 build path 导入jar包，tomcat启动报错找不到该类...
在eclipse集成tomcat开发java web项目时,引入的外部jar包,编译通过,但启动tomcat运行web时提示找不到jar包内的类,需要作如下配置,将jar包在部署到集成的tomcat环 ...
eclipse找不到java方法,Linux启动eclipse报错找不到java环境解决方法
在Linux mint下,前几天还用得很好的的eclipse,今天开机不知为什么这样. Eclipse 3.6 在 linux mint 12 可以在终端顺利启动Eclipse,但是鼠标双击,或者用起 ...
操作canal，报错：Caused by: java.io.IOException: end of stream when reading header
报错:Caused by: java.io.IOException: end of stream when reading header 解决: canal连不上,报错:原因端口变更配置文件(cana ...
【错误记录】Android Studio 编译报错 ( Could not determine java version from ‘11.0.8‘. ② | 升级 Gradle 版本 )
文章目录一.报错信息二.解决方案解决 Could not determine java version from '11.0.8'. 问题 , 有两种方案 , 方案一 : Android Stu ...
Tomcat项目报错：eval: /usr/java/bin/java: not found
报错信息 : eval: /usr/java/bin/java: not found 解决办法 1.: 1.查找java路径: # which java /usr/lib/jvm/jdk-10.0.1 ...
DataX踩坑2 | 定时任务crontab不执行或报错：/bin/sh: java: command not found
前面两天写了一个DataX的增量同步脚本,今天检查了一下发现定时任务没有执行成功,数据并没有同步.以下为排查问题和解决方法. 一.定时任务crontab不执行脚本(测试用的可以设为每分钟一次:*/1 ...
Unity 之发布 Mac 报错 il2cppcore.dll did not run propertly
Unity 之发布Mac报错il2cppcore.dll did not run propertly 一,遇到问题二,尝试解决 2.1 方案一:打包Mono 2.2 方案二:删除引用库 2.3 方 ...
IDEA点击download source 报错：Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.0.1
IDEA点击Download sources 报错:Caused by: java.rmi.ConnectException: Connection refused to host: 127.0.0. ...
报错：Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'USER_NICK_NAME' in
推荐阅读:报错原因详解报错信息 Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'USER_ ...

HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))

HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))相关推荐

最新文章

热门文章