使用JMX监控Kafka
使用JMX监控Kafka
版权声明:本文为博主原创文章,未经博主允许不得转载。
目录(?)[+]
开启JMX端口
- if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
- export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
- export JMX_PORT="9999"
- fi
通过Jconsole测试时候可以连接
通过JavaAPI来访问
- public class KafkaDataProvider{
- protected final Logger LOGGER = LoggerFactory.getLogger(getClass());
- private static final String MESSAGE_IN_PER_SEC = "kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec";
- private static final String BYTES_IN_PER_SEC = "kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec";
- private static final String BYTES_OUT_PER_SEC = "kafka.server:type=BrokerTopicMetrics,name=BytesOutPerSec";
- private static final String PRODUCE_REQUEST_PER_SEC = "kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce";
- private static final String CONSUMER_REQUEST_PER_SEC = "kafka.network:type=RequestMetrics,name=RequestsPerSec,request=FetchConsumer";
- private static final String FLOWER_REQUEST_PER_SEC = "kafka.network:type=RequestMetrics,name=RequestsPerSec,request=FetchFollower";
- private static final String ACTIVE_CONTROLLER_COUNT = "kafka.controller:type=KafkaController,name=ActiveControllerCount";
- private static final String PART_COUNT = "kafka.server:type=ReplicaManager,name=PartitionCount";
- public String extractMonitorData() {
- //TODO 通过调用API获得IP以及参数
- KafkaRoleInfo monitorDataPoint = new KafkaRoleInfo();
- String jmxURL = "service:jmx:rmi:///jndi/rmi://192.168.40.242:9999/jmxrmi";
- try {
- MBeanServerConnection jmxConnection = MetricDataUtils.getMBeanServerConnection(jmxURL);
- ObjectName messageCountObj = new ObjectName(MESSAGE_IN_PER_SEC);
- ObjectName bytesInPerSecObj = new ObjectName(BYTES_IN_PER_SEC);
- ObjectName bytesOutPerSecObj = new ObjectName(BYTES_OUT_PER_SEC);
- ObjectName produceRequestsPerSecObj = new ObjectName(PRODUCE_REQUEST_PER_SEC);
- ObjectName consumerRequestsPerSecObj = new ObjectName(CONSUMER_REQUEST_PER_SEC);
- ObjectName flowerRequestsPerSecObj = new ObjectName(FLOWER_REQUEST_PER_SEC);
- ObjectName activeControllerCountObj = new ObjectName(ACTIVE_CONTROLLER_COUNT);
- ObjectName partCountObj = new ObjectName(PART_COUNT);
- Long messagesInPerSec = (Long) jmxConnection.getAttribute(messageCountObj, "Count");
- Long bytesInPerSec = (Long) jmxConnection.getAttribute(bytesInPerSecObj, "Count");
- Long bytesOutPerSec = (Long) jmxConnection.getAttribute(bytesOutPerSecObj, "Count");
- Long produceRequestCountPerSec = (Long) jmxConnection.getAttribute(produceRequestsPerSecObj, "Count");
- Long consumerRequestCountPerSec = (Long) jmxConnection.getAttribute(consumerRequestsPerSecObj, "Count");
- Long flowerRequestCountPerSec = (Long) jmxConnection.getAttribute(flowerRequestsPerSecObj, "Count");
- Integer activeControllerCount = (Integer) jmxConnection.getAttribute(activeControllerCountObj, "Value");
- Integer partCount = (Integer) jmxConnection.getAttribute(partCountObj, "Value");
- monitorDataPoint.setMessagesInPerSec(messagesInPerSec);
- monitorDataPoint.setBytesInPerSec(bytesInPerSec);
- monitorDataPoint.setBytesOutPerSec(bytesOutPerSec);
- monitorDataPoint.setProduceRequestCountPerSec(produceRequestCountPerSec);
- monitorDataPoint.setConsumerRequestCountPerSec(consumerRequestCountPerSec);
- monitorDataPoint.setFlowerRequestCountPerSec(flowerRequestCountPerSec);
- monitorDataPoint.setActiveControllerCount(activeControllerCount);
- monitorDataPoint.setPartCount(partCount);
- } catch (IOException e) {
- e.printStackTrace();
- } catch (MalformedObjectNameException e) {
- e.printStackTrace();
- } catch (AttributeNotFoundException e) {
- e.printStackTrace();
- } catch (MBeanException e) {
- e.printStackTrace();
- } catch (ReflectionException e) {
- e.printStackTrace();
- } catch (InstanceNotFoundException e) {
- e.printStackTrace();
- }
- return monitorDataPoint.toString();
- }
- public static void main(String[] args) {
- System.out.println(new KafkaDataProvider().extractMonitorData());
- }
- /**
- * 获得MBeanServer 的连接
- *
- * @param jmxUrl
- * @return
- * @throws IOException
- */
- public MBeanServerConnection getMBeanServerConnection(String jmxUrl) throws IOException {
- JMXServiceURL url = new JMXServiceURL(jmxUrl);
- JMXConnector jmxc = JMXConnectorFactory.connect(url, null);
- MBeanServerConnection mbsc = jmxc.getMBeanServerConnection();
- return mbsc;
- }
- }
其他工具
除了自己编写定制化的监控程序外
kafka-web-console
https://github.com/claudemamo/kafka-web-console
部署sbt:
http://www.scala-sbt.org/0.13/tutorial/Manual-Installation.html
http://www.scala-sbt.org/release/tutorial/zh-cn/Installing-sbt-on-Linux.html
KafkaOffsetMonitor
https://github.com/quantifind/KafkaOffsetMonitor/releases/tag/v0.2.0
java -cp KafkaOffsetMonitor-assembly-0.2.0.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk localhost:12181 --port 8080 --refresh 5.minutes --retain 1.day
Mx4jLoader
使用JMX监控Kafka相关推荐
- 【kafka】JMX 监控kafka FINER RMI TCP getConnectionId IOException
1.背景 使用Jmx监控kafka相关信息,但是运行的时候报错如下 我的代码大致逻辑是 JMXServiceUrl jmx = new JMXServiceUrl(url) JMXConnector ...
- 【kafka】JMX 监控kafka kafka rmi NoSuchObjectException no such object in table
1.背景 使用Jmx监控kafka相关信息,但是运行的时候报错如下 我的代码大致逻辑是 JMXServiceUrl jmx = new JMXServiceUrl(url) JMXConnector ...
- 【kafka】java使用jmx 监控Kafka
1.概述 想使用java jmx监控kafka,关于jmx相关的概念请参考 [Java]java jmx 入门案例 进阶版参考:[Spring]SpringBoot 如何使用JMX 2.kafkal开 ...
- 如何使用JMX监控Kafka
欢迎支持笔者新作:<深入理解Kafka:核心设计与实践原理>和<RabbitMQ实战指南>,同时欢迎关注笔者的微信公众号:朱小厮的博客. 欢迎跳转到本文的原文链接:https: ...
- Kafka JMX 监控 之 jmxtrans + influxdb + grafana
目录 效果图 环境准备 安装 influxdb 安装我们刚刚下载 influxdb rpm文件 查看默认配置 修改参数 启动 influxdb 查看启动状态 设置基本配置 influxdb 其他命令扩 ...
- 【Kafka】Window下kafka开启JMX监控
1.概述 因为需要,需要在windows下开启kafka,然后kafka开启JMX监控 同样是修改kafka-server-start文件,但是修改的是kafka-server-start.bat I ...
- gc日志怎么看_你应该怎么监控Kafka?
监控 Kafka,历来都是个老大难的问题.无论是在我维护的微信公众号,还是 Kafka QQ群里面,大家问得最多的问题,一定是 Kafka 的监控.大家提问的内容看似五花八门,但真正想了解的,其实都是 ...
- Prometheus监控Kafka集群
prometheus监控kafka常见的有两种开源方案,一种是传统的部署exporter的方式,一种是通过jmx配置监控,本文将采用第二种方式进行配置. 项目地址: kafka_exporter:ht ...
- 10 Confluent_Kafka权威指南 第十章:监控kafka
文章目录 CHAPTER 10 Monitoring Kafka 监控kafka Metric Basics 度量基础 Where Are the Metrics? 度量标准在哪? Internal ...
最新文章
- CKEditor的安装与基本使用(JSP)
- Ethernet/IP 学习笔记二
- 一些Xcode快捷键,给新手
- GM6 frontend Shell._resolveHashFragment when tile is clicked
- code point,code unit
- 系统防止绕过程序直接数据库修改数据(金额等敏感数据)
- hdu4410(Boomerang)(计算几何)
- 看看如何有效地在你的设计工作中使用霓虹色?
- LinkButton中添加删除确认框
- 360 绑架浏览器首页的解决方法
- 基于python的OCR中文识别教程
- 图像预处理——对数变换
- 无法听到计算机发出的声音怎么办,qq语音听不到对方的声音怎么办_电脑qq语音我听不到对方声音修复方法-win7之家...
- 计量模型 | 时间固定效应与时间趋势项
- E9000刀片服务器维护记录
- LoadRunner-登陆web tours订票网站,预订一张机票后退出-1
- 不谋全局者 , 不足以谋一域 : 不谋万世者 , 不足以谋一时
- 17种Vue适用于移动端的ui框架
- 谈谈我对京东的认识(2):商业价值和前景分析
- USACO Farmer_John收苹果 (dp)
热门文章
- H3CTE京东翰林讲师分享实验2 网络设备基本调试
- apache workprefork
- ASP.NET AJAX 1.0 发布
- TVM: Deep Learning模型的优化编译器(强烈推荐, 附踩坑记录)
- 每日一淘完成1.3亿美元融资,元生资本、DCM领投,SIG联合投资
- 分布式通信框架 - rmi
- 看完让你彻底搞懂Websocket原理
- Oracle 10g RAC修改IP/VIP地址示例
- 有了jsRender,妈妈再也不用担心我用jq拼接DOM拼接的一团糟了、页面整齐了、其他伙伴读代码也不那么费劲了...
- 零距离感受2015年安全***大赛:静动之美 ***兼备