【Kafka】ZK、Kafka以及EFAK的安装、配置

一、Zookeeper

版本：apache-zookeeper-3.8.0

配置：

zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/root/zookeeper/data
clientPort=2181
maxClientCnxns=60
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
4lw.commands.whitelist=*metricsProvider.className=org.apache.zookeeper.metrics.prometheus.PrometheusMetricsProvider
metricsProvider.httpHost=192.168.223.200
metricsProvider.httpPort=17000
metricsProvider.exportJvmInfo=true

zkServer.sh

第 1 处修改：将 243 行clientPortAddress="localhost"修改如下
clientPortAddress="192.168.223.200"第 2 处修改：将 283 行clientPortAddress="localhost"修改如下
clientPortAddress="192.168.223.200"

启动：

[root@centos-7 bin]# ./zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /root/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED[root@centos-7 bin]# ./zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /root/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: standalone

测试“四字真言”

# echo stat| nc 192.168.223.200 2181
Zookeeper version: 3.8.0-5a02a05eddb59aee6ac762f7ea82e92a68eb9c0f, built on 2022-02-25 08:49 UTC
Clients:/192.168.223.200:46818[1](queued=0,recved=2,sent=2)/192.168.223.200:46830[1](queued=0,recved=5,sent=5)/192.168.223.200:46838[1](queued=0,recved=5,sent=5)/192.168.223.200:46810[1](queued=0,recved=5,sent=5)/192.168.223.200:46828[1](queued=0,recved=4,sent=4)/192.168.223.200:46826[1](queued=0,recved=2,sent=2)/192.168.223.200:46906[0](queued=0,recved=1,sent=0)/192.168.223.200:46824[1](queued=0,recved=5,sent=5)/192.168.223.200:46812[1](queued=0,recved=3,sent=3)/192.168.223.200:46816[1](queued=0,recved=5,sent=5)/192.168.223.200:46834[1](queued=0,recved=5,sent=5)/192.168.223.200:46814[1](queued=0,recved=4,sent=4)/192.168.223.200:46836[1](queued=0,recved=5,sent=5)/192.168.223.200:46820[1](queued=0,recved=4,sent=4)/192.168.223.200:46806[1](queued=0,recved=5,sent=5)/192.168.223.200:46822[1](queued=0,recved=4,sent=4)/192.168.223.200:46808[1](queued=0,recved=3,sent=3)/192.168.223.200:46832[1](queued=0,recved=3,sent=3)Latency min/avg/max: 0/4.8654/66
Received: 76
Sent: 79
Connections: 18
Outstanding: 0
Zxid: 0xd0
Mode: standalone
Node count: 53

其他：

stat 查看状态信息
ruok 查看zookeeper是否启动
dump 列出没有处理的节点，临时节点
conf 查看服务器配置
cons 显示连接到服务端的信息
envi 显示环境变量信息
mntr 查看zk的健康信息
wchs 展示watch的信息
wchc和wchp 显示session的watch信息 path的watch信息

二、Kafka

版本：kafka_2.12-3.1.0

配置：

server.properties

broker.id=0listeners=PLAINTEXT://192.168.223.200:9092
advertised.listeners=PLAINTEXT://192.168.223.200:9092num.network.threads=3
num.io.threads=8socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600log.dirs=/root/kafka/logsnum.partitions=1
num.recovery.threads.per.data.dir=1offsets.topic.replication.factor=1transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1log.flush.interval.messages=10000
log.flush.interval.ms=1000
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000zookeeper.connect=192.168.223.200:2181
zookeeper.connection.timeout.ms=60000group.initial.rebalance.delay.ms=0

kafka-server-start.sh

if [ $# -lt 1 ];
thenecho "USAGE: $0 [-daemon] server.properties [--override property=value]*"exit 1
fi
base_dir=$(dirname $0)if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; thenexport KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties"
fiif [ "x$KAFKA_HEAP_OPTS" = "x" ]; thenexport KAFKA_HEAP_OPTS="-Xmx2G -Xms2G -XX:MetaspaceSize=1G -XX:MaxMetaspaceSize=1G"
fiexport JMX_PORT="9010"EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'}COMMAND=$1
case $COMMAND in-daemon)EXTRA_ARGS="-daemon "$EXTRA_ARGSshift;;*);;
esacexec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"

启动脚本：

cd /root/kafka/bin
./kafka-server-start.sh -daemon ../config/server.properties
tail -f ../logs/server.log

三、EFAK

版本：kafka-eagle-bin-2.1.0

配置：

system-config.properties

efak.zk.cluster.alias=cluster1
cluster1.zk.list=192.168.223.200:2181cluster1.zk.acl.enable=false
cluster1.zk.acl.schema=digest
cluster1.zk.acl.username=test
cluster1.zk.acl.password=test123cluster1.efak.broker.size=20kafka.zk.limit.size=16efak.webui.port=8048efak.distributed.enable=false
efak.cluster.mode.status=master
efak.worknode.master.host=localhost
efak.worknode.port=8085cluster1.efak.offset.storage=kafkacluster1.efak.jmx.uri=service:jmx:rmi:///jndi/rmi://%s/jmxrmiefak.metrics.charts=true
efak.metrics.retain=15efak.sql.topic.records.max=5000
efak.sql.topic.preview.records.max=10efak.topic.token=keadminefak.driver=org.sqlite.JDBC
efak.url=jdbc:sqlite:/root/eagle/web/db/ke.db
efak.username=root
efak.password=YOUR_PASSWORD

启动：

# cd /root/eagle/web/bin && ./ke.sh start...created: META-INF/maven/created: META-INF/maven/org.smartloli.kafka.eagle/created: META-INF/maven/org.smartloli.kafka.eagle/efak-web/inflated: META-INF/maven/org.smartloli.kafka.eagle/efak-web/pom.xmlinflated: META-INF/maven/org.smartloli.kafka.eagle/efak-web/pom.properties
[2022-03-16 22:36:04] INFO: Port Progress: [##################################################] | 100%
[2022-03-16 22:36:08] INFO: Config Progress: [##################################################] | 100%
[2022-03-16 22:36:11] INFO: Startup Progress: [##################################################] | 100%
[2022-03-16 22:36:01] INFO: Status Code[0]
[2022-03-16 22:36:01] INFO: [Job done!]
Welcome to______    ______    ___     __ __/ ____/   / ____/   /   |   / //_// __/     / /_      / /| |  / ,<   / /___    / __/     / ___ | / /| |
/_____/   /_/       /_/  |_|/_/ |_|
( Eagle For Apache Kafka® )Version 2.1.0 -- Copyright 2016-2022
*******************************************************************
* EFAK Service has started success.
* Welcome, Now you can visit 'http://192.168.223.200:8048'
* Account:admin ,Password:123456
*******************************************************************
* <Usage> ke.sh [start|status|stop|restart|stats] </Usage>
* <Usage> https://www.kafka-eagle.org/ </Usage>
*******************************************************************

备注：进程启动成功后，访问EFAK的管理页面，需要等很久，不知道什么原因。

EFAK（Eagel for Apache Kafka）的WEB界面：

【Kafka】ZK、Kafka以及EFAK的安装、配置相关推荐

Web基础配置篇（十四）: Kafka单机、集群的安装配置及使用
Web基础配置篇(十四): Kafka单机.集群的安装配置及使用一.概述 Apache Kafka是一个分布式发布 - 订阅消息系统和一个强大的队列,可以处理大量的数据,并使您能够将消息从一个端点传 ...
【Centos】EFAK（kafka-eagle）对ZK、Kafka可视化管理工具容器化安装与配置
[Centos]EFAK(kafka-eagle)对ZK.Kafka可视化管理工具容器化安装与配置前言构建 Dockerfile system-config.properties works re ...
Kafka：ZK+Kafka+Spark Streaming集群环境搭建（九）安装kafka_2.11-1.1.0
如何搭建配置centos虚拟机请参考<Kafka:ZK+Kafka+Spark Streaming集群环境搭建(一)VMW安装四台CentOS,并实现本机与它们能交互,虚拟机内部实现可以上网.& ...
Kafka：ZK+Kafka+Spark Streaming集群环境搭建（二十一）NIFI1.7.1安装
一.nifi基本配置 1. 修改各节点主机名,修改/etc/hosts文件内容. 192.168.0.120master192.168.0.121slave1192.168.0.122 slave2 ...
Linux环境Kafka安装配置
Linux环境Kafka安装配置 1. 认识Kafa (1) Kafa介绍开源消息系统官网:kafka.apache.org/ 用途:在流式计算中,Kafka一般用来缓存数据,Storm通过消费K ...
Kafka：ZK+Kafka+Spark Streaming集群环境搭建（十二）VMW安装四台CentOS，并实现本机与它们能交互，虚拟机内部实现可以上网。...
Centos7出现异常:Failed to start LSB: Bring up/down networking. 按照<Kafka:ZK+Kafka+Spark Streaming集群环境搭 ...
安装配置kafka集群
1.配置虚拟机zookeeper集群见上一篇博客 2.下载安装Kafka 0 )官方下载地址:http://kafka.apache.org/downloads.html 1 )下载解压安装包 wg ...
阿里云安装配置kafka
1· 概述 1.1 kafka是什么在流式计算中,Kafka一般用来缓存数据,Storm通过消费Kafka的数据进行计算. 1)Apache Kafka是一个开源消息系统,由Scala写成.是由Ap ...
Kafka 安装配置及快速入门
2019独角兽企业重金招聘Python工程师标准>>> 一.简介官网:http://kafka.apache.org/ Apache Kafka是分布式发布-订阅消息系统.它最初由 ...

【Kafka】ZK、Kafka以及EFAK的安装、配置

【Kafka】ZK、Kafka以及EFAK的安装、配置相关推荐

最新文章

热门文章