【kafka Manager 编译教程】【编译环境：Centos7】

KafkaManager下载

【2.0以下版本存在 JQuery漏洞，3.0.0.1+依赖zk3.5+，所以选用3.0.0.0，适配现网zk3.4.10】

【官方地址】https://github.com/yahoo/CMAK【版本】【3.0.0.2开始已提供编译版本】CMAK-3.0.0.0.zip

Sbt下载

【编译环境按照kafka manager的要求下载】

【官方地址】https://www.scala-sbt.org/download.html【版本】sbt 1.3.8 (.zip)

Jdk11下载

【3.0+ 依赖 jdk11，编译环境也必须配套Jdk11】

【官方地址】https://www.oracle.com/java/technologies/javase-jdk11-downloads.html【版本】jdk-11.0.7_linux-x64_bin.tar.gz

配置Jdk11

【3.0+ 依赖 jdk11】

【解压】# tar zxvf jdk-11.0.7_linux-x64_bin.tar.gz -C /home/java11【环境变量】
# vim ~/.bashrc#jdk#
export JAVA_HOME=/home/java11
export PATH=$JAVA_HOME/bin:$PATH【检查】
# source ~/.bashrc
# java -version

检查sbt版本

【检查kafka Manager依赖的sbt版本】

【解压压缩包】
# unzip CMAK-3.0.0.0.zip
# vim CMAK-3.0.0.0/sbt[可见该版本依赖1.3.8版本的sbt]
declare -r sbt_release_version="1.3.8"

安装sbt

【解压】
# unzip -q -d /usr/share/ sbt-1.3.8.zip
# cd /usr/share/sbt/【启动脚本】
# vim /usr/share/sbt/sbt
#!/bin/bash
BT_OPTS="-Xms2048M -Xmx4096M -Xss1M -XX:+CMSClassUnloadingEnabled -XX:MaxPermSize=256M"
java $SBT_OPTS -jar /usr/share/sbt/bin/sbt-launch.jar "$@"
# chmod u+x /usr/share/sbt/sbt【环境变量】
# vim ~/.bashrc
#SBT#
export SBT_HOME=/usr/share/sbt
export PATH=$PATH:$SBT_HOME/bin# source ~/.bashrc

修改 sbt-launch.jar

【解包修改】
# cd /usr/share/sbt/bin/
# mkdir tmpd;cd tmpd
# cp ../sbt-launch.jar .
# mv ../sbt-launch.jar ../sbt-launch.jar.bak
# unzip -q sbt-launch.jar
# mkdir conf
# vim conf/repo.properties
[repositories]localaliyun: http://maven.aliyun.com/nexus/content/groups/public/central: http://repo1.maven.org/maven2/【打包jar】
# jar -cfM ./sbt-launch.jar .
# chmod a+x sbt-launch.jar
# mv sbt-launch.jar .. && cd ../
# rm -rf tmpd

下载sbt依赖

# sbt versioncopying runtime jar...
[info] [launcher] getting org.scala-sbt sbt 1.3.8  (this may take some time)...
waitting ......【失败则修改源后重试，下载完后重新执行命令测试】
【如果下载卡在那个包，已经没有下行网速了，则ctrl+c退出后重新下载】
【修改源】
# vim ~/.sbt/repositories
[repositories]
local
aliyun: http://maven.aliyun.com/nexus/content/groups/public/
typesafe: http://repo.typesafe.com/typesafe/ivy-releases/, [organization]/[module]/(scala_[scalaVersion]/)(sbt_[sbtVersion]/)[revision]/[type]s/[artifact](-[classifier]).[ext], bootOnly
sonatype-oss-releases
maven-central
sonatype-oss-snapshots

编译KafkaManager

# cd /home/CMAK-3.0.0.0
# ./sbt clean dist编译完成后会在：/home/kafkamanager/CMAK-3.0.0.0/target/universal 生成安装包 cmak-3.0.0.0.zip

安装KfkaManager

【解压】
# unzip -q -d /home/alpha cmak-3.0.0.0.zip
# mv cmak-3.0.0.0 cmak

目录结构

【目录结构】

【项目】
cmak/
├── bin
├── conf
├── java11
├── lib
├── README.md
├── share
├── start.sh
└── stop.sh【日志】
/data01/alpha/cmak/logs/
└── application.log

部署目录

/home/alpha/cmak

服务端口

Port：9000

配置服务

【登陆鉴权】

$ vim conf/application.conf
【1】
kafka-manager.zkhosts="10.66.40.97:3181,10.66.40.98:3181,10.66.40.99:3181"
【2】
basicAuthentication.enabled=true
【3】
akka {loggers = ["akka.event.slf4j.Slf4jLogger"]loglevel = "OFF"
}
【4】
basicAuthentication.password="Jet#2dg%S@A"

日志配置

【默认关闭所有日志，异常时日志会很多】

【=================== logger.xml ==========================】
$ vim conf/logger.xml<configuration><conversionRule conversionWord="coloredLevel" converterClass="play.api.Logger$ColoredLevel" /><appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"><file>/data01/alpha/cmak/logs/application.log</file><encoder><pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern></encoder><rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"><fileNamePattern>/data01/alpha/cmak/logs/application.%d{yyyy-MM-dd}.log</fileNamePattern><maxHistory>3</maxHistory><totalSizeCap>100MB</totalSizeCap></rollingPolicy></appender><logger name="play" level="OFF" /><logger name="application" level="OFF" /><!-- Off these ones as they are annoying, and anyway we manage configuration ourself --><logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" /><logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" /><logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" /><logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" /><logger name="org.apache.zookeeper.ClientCnxn" level="OFF" /><logger name="org.apache.zookeeper" level="OFF"/><logger name="org.apache.curator.ConnectionState" level="OFF"/><logger name="kafka.manager.KafkaManager" level="OFF"/><logger name="akka" level="OFF" /><logger name="kafka" level="OFF" /><root level="OFF"><appender-ref ref="FILE" /></root></configuration>【=================== logback.xml ==========================】
$ vim conf/logback.xml
<!--~ Copyright (C) 2009-2015 Typesafe Inc. <http://www.typesafe.com>-->
<!-- The default logback configuration that Play uses if no other configuration is provided -->
<configuration><appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender"><file>/data01/alpha/cmak/logs/application.log</file><encoder><pattern>%date - [%level] - from %logger in %thread %n%message%n%xException%n</pattern></encoder><rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"><fileNamePattern>/data01/alpha/cmak/logs/application.%d{yyyy-MM-dd}.log</fileNamePattern><maxHistory>3</maxHistory><totalSizeCap>100MB</totalSizeCap></rollingPolicy></appender><appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"><encoder><pattern>%date - [%level] %logger{15} - %message%n%xException{10}</pattern></encoder></appender><appender name="ASYNCFILE" class="ch.qos.logback.classic.AsyncAppender"><appender-ref ref="FILE" /></appender><appender name="ASYNCSTDOUT" class="ch.qos.logback.classic.AsyncAppender"><appender-ref ref="STDOUT" /></appender><logger name="play" level="OFF" /><logger name="application" level="OFF" /><logger name="kafka.manager" level="OFF" /><!-- Off these ones as they are annoying, and anyway we manage configuration ourself --><logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" /><logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" /><logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" /><logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" /><logger name="org.apache.zookeeper" level="OFF"/><logger name="org.apache.curator.ConnectionState" level="OFF"/><root level="OFF"><appender-ref ref="ASYNCFILE" /><appender-ref ref="ASYNCSTDOUT" /></root></configuration>

存储目录

【创建日志目录】

$ mkdir -p /data01/alpha/cmak/logs

服务管理

【启动脚本】

$ vim cmak/start.sh
#!/bin/bash
if [ -f RUNNING_PID ];then echo -e "Delete RUNNING_PID !\n";rm RUNNING_PID;fi
/home/alpha/cmak/bin/cmak -Dconfig.file=/home/alpha/cmak/conf/application.conf -java-home /home/alpha/cmak/java11 -Dhttp.port=9000 2>&1 >/dev/null &

【停止脚本】

$ cat cmak/stop.sh
#!/bin/bash
jps|grep -w 'ProdServerStart'|awk '{print $1}'|xargs kill -9
if [ -f RUNNING_PID ];then echo -e "Delete RUNNING_PID !\n";rm RUNNING_PID;fi

启动服务

【启动】
$ cd ~/cmak
$ ./start.sh【检查】
$ jps
1234 ProdServerStart

登陆页面

【web控制台】
http://10.66.40.97:9000【用户名/密码】
admin / Jet#2dg%S@A

新建集群

【WEB创建集群】

配置项	参数
Cluster Name	lnyd-1
Cluster Zookeeper Hosts	10.66.40.97:3181,10.66.40.98:3181,10.66.40.99:3181
Kafka Version	0.10.1.0
勾选	Enable JMX Polling
勾选	Poll consumer information
勾选	Filter out inactive consumers
勾选	Enable Active OffsetCache
勾选	Display Broker and Topic Size

【注意】勾选“Enable JMX Polling” 后，kafka的启动脚本 “kafka-server-start.sh” 需要增加 “export JMX_PORT=9099”

# vim kafka/bin/kafka-server-start.sh
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; thenexport KAFKA_HEAP_OPTS="-Xmx4G -Xms4G"export JMX_PORT=9099
fi

krb 认证

【配置】
# vim conf/application.conf【kerberos】
-Djava.security.auth.login.config=/etc/ksecurity/my-jaas.conf

cmak(kafka Manager) 编译教程相关推荐

Kafka Manager 编译 + 部署运行
准备条件本地有 sbt 环境 Java 8+ 下载 cd /usr/local/srcgit clone https://github.com/yahoo/kafka-manager 由于编译时需要 ...
最好用的Redis Desktop Manager 0.9.3 版本下载以及源码编译教程
文章目录一.前言二.编译教程 2.1 [redis destop manager 的源码地址](https://github.com/uglide/RedisDesktopManager) 2.2 ...
Apache Kafka-CMAK（kafka manager）安装部署使用
文章目录 Github地址二进制安装包下载 3.0.0.2 及之后 3.0.0.2 之前 Require 安装启动配置 Github地址 https://github.com/yahoo/CMA ...
kafka manager安装和使用
1.GitHub下载kafka manager的源码并且编译链接:https://github.com/yahoo/CMAK. 2.编译命令和编译完成后的文件路径编译命令:sbt clean di ...
CM12同步源码及编译教程
同时提供基于安卓5.0的MKL魔趣猪扒饭编译教程~ [玩机组出品]魔趣猪扒饭MKL50.1编译教程 http://www.oneplusbbs.com/forum.php?mod=viewthread ...
Kafka manager安装 (支持0.10以后版本consumer)
下载地址: https://pan.baidu.com/s/1jIE3YL4 步骤: 1. 解压kafka-manager-1.3.2.1.zip 2. cd kafka-manager-1.3.2 ...
最新版OpenWrt编译教程，解决依赖问题
最新版OpenWrt编译教程,解决依赖问题参考文章: (1)最新版OpenWrt编译教程,解决依赖问题 (2)https://www.cnblogs.com/jzssuanfa/p/7400840. ...
kafka查看topic中的数据_实战！Kafka Manager能统计出Topic中的记录条数吗？
问题描述今天现场实施同事说Kafka Manager上显示有3500w条记录,但使用我们的平台落地后,一统计发现只有2200w条记录,这是不是说明我们的平台存在丢数据的可能. 经了解,对接方是通过如 ...
详解S60 WebKit 21772编译教程
S60 WebKit 21772编译教程是本文要介绍的内容,主要是来了解并学习S60 WebKit应用,具体内容的学习来看本文详解. 1.安装环境: perl jdk S60-SDK-0548-3.0 ...
CEF编译教程（手把手教学版）
CEF编译教程 1.下载.安装CMake工具: 下载链接:https://cmake.org/download/ 选择cmake-3.15.1-win64-x64.msi 下载直接安装即可安装之后命 ...

cmak(kafka Manager) 编译教程