Kafka 入门教程之一: 安装
2024-06-08 20:43:00
<!DOCTYPE html><html><head><title>Kafka 入门教程之一: 安装</title><meta charset='utf-8'><link href='https://cdn.maxiang.io/res-min/themes/marxico.css' rel='stylesheet'><style>
.note-content {font-family: 'Helvetica Neue', Arial, 'Hiragino Sans GB', STHeiti, 'Microsoft YaHei', 'WenQuanYi Micro Hei', SimSun, Song, sans-serif;}
</style></head><body><div id='preview-contents' class='note-content'><h2 id="kafka-入门教程之一-安装">Kafka 入门教程之一: 安装</h2><h4 id="1-官网下载软件">1. 官网下载软件</h4>
<p><a href="http://kafka.apache.org/" target="_blank">http://kafka.apache.org/</a> </p><p><img longdesc="./1543296020347.png" alt="Alt text" title="" type="image/png" src="https://img-blog.csdnimg.cn/2022010705415375303.png" class=""></p><h4 id="2-安装单节点kafka">2. 安装单节点kafka</h4><h5 id="a-创建用户kafka">a) 创建用户kafka</h5><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[root@tjtestrac1 ~]<span class="hljs-comment"># useradd kafka </span>
</div><div class="hljs-line">[root@tjtestrac1 ~]<span class="hljs-comment"># passwd kafka</span>
</div><div class="hljs-line">Changing password <span class="hljs-keyword">for</span> user kafka.
</div><div class="hljs-line">New password:
</div><div class="hljs-line">BAD PASSWORD: The password is shorter than <span class="hljs-number">8</span> characters
</div><div class="hljs-line">Retype new password:
</div><div class="hljs-line">passwd: all authentication tokens updated successfully.
</div></code></pre><h4 id="b-解压软件包">b) 解压软件包</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 ~]$ ls
</div><div class="hljs-line">kafka_2.<span class="hljs-number">12</span>-<span class="hljs-number">2.1</span>.<span class="hljs-number">0</span>.tgz
</div><div class="hljs-line">[kafka@tjtestrac1 ~]$ tar -xvf kafka_2.<span class="hljs-number">12</span>-<span class="hljs-number">2.1</span>.<span class="hljs-number">0</span>.tgz
</div></code></pre><h4 id="c-配置环境变量-kafkahome-和-javahome">c) 配置环境变量 kafka_home 和 java_home</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 ~]$ vi .bash_profile
</div><div class="hljs-line">export KAFKA_HOME=/home/kafka/kafka_2.<span class="hljs-number">12</span>-<span class="hljs-number">2.1</span>.<span class="hljs-number">0</span>
</div><div class="hljs-line">export JAVA_HOME=/u02/cassandra/java/jdk1.<span class="hljs-number">8.0</span>_162
</div><div class="hljs-line">PATH=<span class="hljs-variable">$PATH:</span><span class="hljs-variable">$HOME</span>/.local/bin:<span class="hljs-variable">$HOME</span>/bin:<span class="hljs-variable">$JAVA_HOME</span>/bin:<span class="hljs-variable">$KAFKA_HOME</span>/bin [kafka@tjtestrac1 ~]$ source .bash_profile
</div></code></pre><h4 id="3-启动zookeeper-kafka-组件">3. 启动zookeeper, kafka 组件</h4><h4 id="a-为zookeeper-创建文件夹">a) 为zookeeper 创建文件夹</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ mkdir -p /home/kafka/zk
</div></code></pre><h4 id="b-修改kafka-内置的zk-配置文件">b) 修改kafka 内置的zk 配置文件</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ vi zookeeper.properties
</div><div class="hljs-line">dataDir=/home/kafka/zk
</div><div class="hljs-line"><wbr>
</div><div class="hljs-line"><span class="hljs-comment"># the port at which the clients will connect</span>
</div><div class="hljs-line">clientPort=<span class="hljs-number">2181</span>
</div><div class="hljs-line"><span class="hljs-comment"># disable the per-ip limit on the number of connections since this is a non-production config</span>
</div><div class="hljs-line">maxClientCnxns=<span class="hljs-number">0</span>
</div></code></pre><h4 id="c-启动zk">c) 启动zk</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">kafka@tjtestrac1 config]$ zookeeper-server-start.sh <span class="hljs-variable">$KAFKA_HOME</span>/config/zookeeper.properties &
</div></code></pre><h4 id="d-查看后台进程">d) 查看后台进程</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 ~]$ jps
</div><div class="hljs-line"><span class="hljs-number">30323</span> QuorumPeerMain
</div><div class="hljs-line"><span class="hljs-number">30660</span> Jps
</div></code></pre><h4 id="4-启动kafka">4. 启动kafka</h4><h4 id="a-为kafka-创建文件夹">a) 为kafka 创建文件夹</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 ~]$ mkdir -p /home/kafka/kf
</div></code></pre><h4 id="b-修改kafka-的配置文件">b) 修改kafka 的配置文件</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ vi server.properties
</div><div class="hljs-line">log.dirs=/home/kafka/kf
</div></code></pre><h4 id="c-启动kafka-进程">c) 启动kafka 进程</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ kafka-server-start.sh <span class="hljs-variable">$KAFKA_HOME</span>/config/server.properties &
</div></code></pre><h4 id="d-查看后台进程-1">d) 查看后台进程</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ jps
</div><div class="hljs-line"><span class="hljs-number">30323</span> QuorumPeerMain
</div><div class="hljs-line"><span class="hljs-number">4739</span> Kafka
</div><div class="hljs-line"><span class="hljs-number">5414</span> Jps
</div></code></pre><h4 id="5-创建一个主题topic">5. 创建一个主题topic</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ kafka-topics.sh --create --zookeeper localhost:<span class="hljs-number">2181</span> --replication-factor <span class="hljs-number">1</span> --partitions <span class="hljs-number">1</span> --topic test
</div><div class="hljs-line"><wbr>
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">27</span>:<span class="hljs-number">47</span>,<span class="hljs-number">517</span>] INFO Accepted socket connection from /<span class="hljs-number">127.0</span>.<span class="hljs-number">0.1</span>:<span class="hljs-number">58228</span> (org.apache.zookeeper.server.NIOServerCnxnFactory)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">27</span>:<span class="hljs-number">47</span>,<span class="hljs-number">519</span>] INFO Client attempting to establish new session at /<span class="hljs-number">127.0</span>.<span class="hljs-number">0.1</span>:<span class="hljs-number">58228</span> (org.apache.zookeeper.server.ZooKeeperServer)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">27</span>:<span class="hljs-number">47</span>,<span class="hljs-number">521</span>] INFO Established session <span class="hljs-number">0</span>x10052ce304e0001 with negotiated timeout <span class="hljs-number">30000</span> <span class="hljs-keyword">for</span> client /<span class="hljs-number">127.0</span>.<span class="hljs-number">0.1</span>:<span class="hljs-number">58228</span> (org.apache.zookeeper.server.ZooKeeperServer)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">27</span>:<span class="hljs-number">47</span>,<span class="hljs-number">802</span>] INFO Got user-level KeeperException when processing sessionid:<span class="hljs-number">0</span>x10052ce304e0001 type:setData cxid:<span class="hljs-number">0</span>x4 zxid:<span class="hljs-number">0</span>x1f txntype:-<span class="hljs-number">1</span> reqpath:n/a Error Path:/config/topics/test Error:KeeperErrorCode = NoNode <span class="hljs-keyword">for</span> /config/topics/test (org.apache.zookeeper.server.PrepRequestProcessor)
</div><div class="hljs-line">Created topic <span class="hljs-string">"test"</span>.
</div></code></pre><h4 id="列出主题">列出主题:</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 config]$ kafka-topics.sh --list --zookeeper localhost:<span class="hljs-number">2181</span>
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">29</span>:<span class="hljs-number">31</span>,<span class="hljs-number">251</span>] INFO Accepted socket connection from /<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">1</span>:<span class="hljs-number">64976</span> (org.apache.zookeeper.server.NIOServerCnxnFactory)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">29</span>:<span class="hljs-number">31</span>,<span class="hljs-number">253</span>] INFO Client attempting to establish new session at /<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">1</span>:<span class="hljs-number">64976</span> (org.apache.zookeeper.server.ZooKeeperServer)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">29</span>:<span class="hljs-number">31</span>,<span class="hljs-number">255</span>] INFO Established session <span class="hljs-number">0</span>x10052ce304e0002 with negotiated timeout <span class="hljs-number">30000</span> <span class="hljs-keyword">for</span> client /<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">1</span>:<span class="hljs-number">64976</span> (org.apache.zookeeper.server.ZooKeeperServer)
</div><div class="hljs-line">test
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">29</span>:<span class="hljs-number">31</span>,<span class="hljs-number">302</span>] INFO Processed session termination <span class="hljs-keyword">for</span> sessionid: <span class="hljs-number">0</span>x10052ce304e0002 (org.apache.zookeeper.server.PrepRequestProcessor)
</div><div class="hljs-line">[<span class="hljs-number">2018</span>-<span class="hljs-number">11</span>-<span class="hljs-number">22</span> <span class="hljs-number">17</span>:<span class="hljs-number">29</span>:<span class="hljs-number">31</span>,<span class="hljs-number">303</span>] INFO Closed socket connection <span class="hljs-keyword">for</span> client /<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">0</span>:<span class="hljs-number">1</span>:<span class="hljs-number">64976</span> which had sessionid <span class="hljs-number">0</span>x10052ce304e0002 (org.apache.zookeeper.server.NIOServerCnxn)
</div></code></pre><h4 id="6-启动生产者来发送数据">6. 启动生产者来发送数据</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line">[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:<span class="hljs-number">9092</span> --topic test
</div><div class="hljs-line">>jason
</div><div class="hljs-line">>comm on
</div><div class="hljs-line">>test message
</div></code></pre><h4 id="7-启动消费者来接收数据">7. 启动消费者来接收数据</h4><pre class="prettyprint hljs-dark"><code class="language-powershell hljs"><div class="hljs-line"> [kafka@tjtestrac1 ~]$ kafka-console-consumer.sh --bootstrap-server localhost:<span class="hljs-number">9092</span> --topic test --from-beginning
</div><div class="hljs-line"><wbr>
</div><div class="hljs-line">[kafka@tjtestrac1 ~]$ kafka-console-producer.sh --broker-list localhost:<span class="hljs-number">9092</span> --topic test
</div><div class="hljs-line">>jason
</div><div class="hljs-line">>comm on
</div><div class="hljs-line">>test message
</div><div class="hljs-line">>next message
</div></code></pre></div></body></html>转载于:https://blog.51cto.com/14092938/2322529
Kafka 入门教程之一: 安装相关推荐
- Kafka入门教程及安装
目录 Kafka消息队列 Kafka基础概念 关于顺序和分区 关于副本 丢消息和重复收到消息 Zookeeper安装教程 安装步骤 zookeeper集群配置 Kafka安装教程 安装步骤 Kafka ...
- Kafka入门教程与详解
1 Kafka入门教程 1.1 消息队列(Message Queue) Message Queue消息传送系统提供传送服务.消息传送依赖于大量支持组件,这些组件负责处理连接服务.消息的路由和传送.持久 ...
- Linux搭建waf防火墙,Linux入门教程:nginx安装配置naxsi waf防火墙,nginxwaf Naxsi 不依赖...
Linux入门教程:nginx安装配置naxsi waf防火墙,nginxwaf Naxsi 不依赖 Naxsi 是第三方 nginx 模块 ,它和 Modsecurity 都是开源 WAF ,但是它 ...
- Kafka入门教程（一）
转自:https://blog.csdn.net/yuan_xw/article/details/51210954 1 Kafka入门教程 1.1 消息队列(Message Queue) Messag ...
- Kafka教程(一)Kafka入门教程
Kafka教程(一)Kafka入门教程 1 Kafka入门教程 1.1 消息队列(Message Queue) Message Queue消息传送系统提供传送服务.消息传送依赖于大量支持组件,这些组件 ...
- mysql 开源入门_入门教程：安装配置新版MySQL 8开源数据库
原标题:入门教程:安装配置新版MySQL 8开源数据库 [ 来自IT168] [IT168 技术]MySQL是现代应用程序编程堆栈中常见的数据库管理系统.如果您想要一个适用于您应用程序的,便于利用丰富 ...
- RabbitMQ入门教程（安装，管理插件，Publisher/Consumer/交换机/路由/队列/绑定关系，及如何保证100%投递等）
RabbitMQ入门教程(安装,管理插件,Publisher/Consumer/交换机/路由/队列/绑定关系,及如何保证100%投递等) 1. RabbitMQ简介及AMQP协议 开源的消息代理和队列 ...
- Kafka : Kafka入门教程和JAVA客户端使用
目录 目录 Kafka简介 环境介绍 术语介绍 消费模式 下载 集群安装配置 命令使用 JAVA实战 参考文献 Kafka简介 由Scala和Java编写,Kafka是一种高吞吐量的分布式发布订阅消息 ...
- Arduino入门教程-软件安装
arduino用作嵌入式入门还是非常好的,门槛相对比较低,学会了转其他的芯片也十分方便.下面就不多说了,直接进入图文式傻瓜教程. 不想看图的可以参考本人视频arduino软件安装_哔哩哔哩_bilib ...
- Kafka 入门教程（超详细）
文章目录 1. Kafka 概述 1.1 定义 1.2 消息队列 1.2.1 传统消息队列的应用场景 1.2.2 消息队列的两种形式 1.3 Kafka 基础架构 2. Kafka 的安装 2.1 安 ...
最新文章
- ICLR2020全析解读：华人学者占据60%，谷歌再次领跑！（附最新高引华人榜单）...
- 枚举与Combox的SelectedIndex绑定出现怪异问题的解决方案
- Android10.0 Binder通信原理(六)-Binder数据如何完成定向打击
- 报告称企业部署Windows 10比此前版本更快
- 丑憨批的html实战
- boss直聘账号异常登不上_【苹果IOS游戏账号分享】据说是最欢乐的2D冒险游戏？Q萌有趣，好不好玩你说了算——惊奇剑士...
- 计算机加一块硬盘,老电脑卡顿不一定没救了 加一块SSD就能焕发新生
- SpringCloud创建Eureka Client服务注册
- Java基础知识编写一个HelloWorld案例
- Silverlight(CodeNameWPF/E) Features一览表
- xampp 403 禁止访问 问题解决
- win10装系统--笔记
- QML Text 文字元素
- Visio 2019 专业版 下载地址
- 一次函数的斜率公式_一次函数斜率公式是什么？
- QT实现点击按钮打开和关闭窗口
- 2019论文选题热点计算机,关于进行2019届毕业论文选题工作的通知
- 使用python制作属于自己的地铁图
- 移动光猫路由改桥接降低延时初尝试
- 语义标签(Semantic label)与多模态模型的一些关系