RocketMQ-初体验RocketMQ(03)_RocketMQ多机集群部署
文章目录
- 环境准备
- 参考配置
- 集群搭建
- 130主机
- broker 配置文件
- 启动namesrv
- 启动broker Master 和 broker Slave
- 131主机
- broker 配置文件
- 启动namesrv
- 启动broker Master 和 broker Slave
- 查看集群监控状态
- 测试发送和消费消息
- 双机互为主备的配置
- RocketMQ4.3.X配置参数
完成了单节点的RocketMQ的安装 RocketMQ-初体验RocketMQ(02)_单节点RocketMQ的安装 ,我们来搞个2个节点的集群来玩下
环境准备
CentOS7
192.168.18.130 、192.168.18.131
Q: 生产环境,假设你有2台主机,应该如何部署RocketMQ集群更HA一些?
A:
你有 A、B两台主机,
A主机 部署 : broker-a 主节点 和 broker-b 从节点
B主机 部署: broker-b 主节点 和 borker-a 从节点
两台主机互为主备,HA更可靠一些,即使有一台主机,假设B主机 宕机了。 我们的A主机上 仍然还有是 broker-a 主节点 和 broker-b 从节点, 这个时候只是 broker-b 只能提供消费消息的能力了,不能写入了。 还是要比 整个 broker-b cluster 因为主机B的宕机而全部不能提供服务了好。
参考配置
在RocketMQ 安装目录 conf目录下,官方提供了一些参考配置
可以集合自己的场景,到对应的目录下看下官方推荐的配置。
集群搭建
130 : brokera-m brokera-s
131: brokerb-m brokerb-s
broker集群在同一台主机上互为主备 。
每台机器上启动一个namesrv。 两个 broker节点,互为主备,构成一个broker集群。
130主机
broker 配置文件
cp 两个文件出来
broker-m.conf master节点的配置文件
broker-s.conf slave节点的配置文件
broker-m.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#集群名称,可自定义
brokerClusterName=DefaultCluster
brokerName=broker‐a
# 0 表示主节点
brokerId=0
# rocketmq‐name服务地址,多个地址用;分开,不配置默认为localhost:9876
namesrvAddr=192.168.18.130:9876;192.168.18.131:9876
deleteWhen=04
fileReservedTime=48
#当前节点角色
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=true
#broker通信端口,默认端口
listenPort=10911
#消息存储根路径
storePathRootDir=/data/rocketmq/store-m
broker-s.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#集群名称,可自定义
brokerClusterName=DefaultCluster
brokerName=broker‐a
# 非0 表示从节点
brokerId=1
# rocketmq‐name服务地址,多个地址用;分开,不配置默认为localhost:9876
namesrvAddr=192.168.18.130:9876;192.168.18.131:9876
deleteWhen=04
fileReservedTime=48
#当前节点角色
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=true
#broker通信端口,默认端口
listenPort=10811
#消息存储根路径
storePathRootDir=/data/rocketmq/store-s
主要注意的地方
- brokerName 主从节点 保持一致
- brokerId 0 表示主节点,从节点 1,2,3…依次类推
- namesrvAddr 配置所有 namesrv的地址,多个地址 使用 ; 分开
- brokerRole 这个要注意 主节点 SYNC_MASTER 或者 ASYNC_MASTER 从节点 SLAVE
- listenPort 确定端口不要重复占用,否则启动失败
启动namesrv
nohup sh bin/mqnamesrv -n 192.168.18.130:9876&
最好通过-n指定主IP,否则的话有可能在多网卡 或者docker环境下 启动失败
[root@artisan rocketmq-all-4.3.2-bin-release]# pwd
/usr/local/rocketmq/rocketmq-all-4.3.2-bin-release
[root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqnamesrv -n 192.168.18.130:9876&
[1] 9236
[root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’
tail -f nohup.out 日志如下:
Java HotSpot(TM) 64-Bit Server VM warning: Using the DefNew young collector with the CMS collector is deprecated and will likely be removed in a future release
Java HotSpot(TM) 64-Bit Server VM warning: UseCMSCompactAtFullCollection is deprecated and will likely be removed in a future release.
The Name Server boot success. serializeType=JSON
查看进程
[root@artisan rocketmq-all-4.3.2-bin-release]# jps |grep -v Jps
9239 NamesrvStartup
[root@artisan rocketmq-all-4.3.2-bin-release]#
启动broker Master 和 broker Slave
主节点: nohup sh bin/mqbroker -c conf/broker-m.conf &
从节点: nohup sh bin/mqbroker -c conf/broker-s.conf &
[root@artisan rocketmq-all-4.3.2-bin-release]# >nohup.out 主节点启动 [root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -c conf/broker-m.conf &
[2] 9415
[root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’
tailf nohup.out
The broker[brokerâa, 192.168.18.130:10911] boot success. serializeType=JSON and name server is 192.168.18.130:9876;192.168.18.131:9876^C^C 从节点启动
[root@artisan rocketmq-all-4.3.2-bin-release]# nohup sh bin/mqbroker -c conf/broker-s.conf &
[3] 9495
[root@artisan rocketmq-all-4.3.2-bin-release]# nohup: ignoring input and appending output to ‘nohup.out’
tailf nohup.out
The broker[brokerâa, 192.168.18.130:10911] boot success. serializeType=JSON and name server is 192.168.18.130:9876;192.168.18.131:9876The broker[brokerâa, 192.168.18.130:10811] boot success. serializeType=JSON and name server is 192.168.18.130:9876;192.168.18.131:9876^C 查看进程
[root@artisan rocketmq-all-4.3.2-bin-release]# jps |grep -v Jps
9239 NamesrvStartup
9419 BrokerStartup
9499 BrokerStartup
[root@artisan rocketmq-all-4.3.2-bin-release]#
131主机
broker 配置文件
操作同130 , 唯一不同的就是这个 brokerName 。
这里贴下 配置
主节点
[root@artisan conf]# cat broker-m.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#集群名称,可自定义
brokerClusterName=DefaultCluster
brokerName=broker‐b
# 0 表示主节点
brokerId=0
# rocketmq‐name服务地址,多个地址用;分开,不配置默认为localhost:9876
namesrvAddr=192.168.18.130:9876;192.168.18.131:9876
deleteWhen=04
fileReservedTime=48
#当前节点角色
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=true
#broker通信端口,默认端口
listenPort=10911
#消息存储根路径
storePathRootDir=/data/rocketmq/store-m
从节点
[root@artisan conf]# cat broker-s.conf
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.#集群名称,可自定义
brokerClusterName=DefaultCluster
brokerName=broker‐b
# 非0 表示从节点
brokerId=1
# rocketmq‐name服务地址,多个地址用;分开,不配置默认为localhost:9876
namesrvAddr=192.168.18.130:9876;192.168.18.131:9876
deleteWhen=04
fileReservedTime=48
#当前节点角色
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH
autoCreateTopicEnable=true
#broker通信端口,默认端口
listenPort=10811
#消息存储根路径
storePathRootDir=/data/rocketmq/store-s
[root@artisan conf]#
启动namesrv
同130
启动broker Master 和 broker Slave
同130
查看集群监控状态
mqadmin clusterlist -n 192.168.18.130:9876
[root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/mqadmin clusterlist -n 192.168.18.130:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name #Broker Name #BID #Addr #Version #InTPS(LOAD) #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
DefaultCluster brokerâa 0 192.168.18.130:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1331
DefaultCluster brokerâa 1 192.168.18.130:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1331
DefaultCluster brokerâb 0 192.168.18.131:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1654
DefaultCluster brokerâb 1 192.168.18.131:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1654
^C^C^C[root@artisan rocketmq-all-4.3.2-bin-release]# ^C
[root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/mqadmin clusterlist -n 192.168.18.131:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name #Broker Name #BID #Addr #Version #InTPS(LOAD) #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
DefaultCluster brokerâa 0 192.168.18.130:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1331
DefaultCluster brokerâa 1 192.168.18.130:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1331
DefaultCluster brokerâb 0 192.168.18.131:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1654
DefaultCluster brokerâb 1 192.168.18.131:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436943.73 0.1654
[root@artisan rocketmq-all-4.3.2-bin-release]#
如果碰到了如下错误
[root@artisan bin]# ./mqadmin clusterlist -n 192.168.18.130:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
org.apache.rocketmq.tools.command.SubCommandException: ClusterListSubCommand command failedat org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:93)at org.apache.rocketmq.tools.command.MQAdminStartup.main0(MQAdminStartup.java:132)at org.apache.rocketmq.tools.command.MQAdminStartup.main(MQAdminStartup.java:83)
Caused by: org.apache.rocketmq.remoting.exception.RemotingTimeoutException: wait response on the channel <192.168.18.130:9876> timeout, 463(ms)at org.apache.rocketmq.remoting.netty.NettyRemotingAbstract.invokeSyncImpl(NettyRemotingAbstract.java:391)at org.apache.rocketmq.remoting.netty.NettyRemotingClient.invokeSync(NettyRemotingClient.java:374)at org.apache.rocketmq.client.impl.MQClientAPIImpl.getBrokerClusterInfo(MQClientAPIImpl.java:1180)at org.apache.rocketmq.tools.admin.DefaultMQAdminExtImpl.examineBrokerClusterInfo(DefaultMQAdminExtImpl.java:275)at org.apache.rocketmq.tools.admin.DefaultMQAdminExt.examineBrokerClusterInfo(DefaultMQAdminExt.java:222)at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.printClusterBaseInfo(ClusterListSubCommand.java:172)at org.apache.rocketmq.tools.command.cluster.ClusterListSubCommand.execute(ClusterListSubCommand.java:88)... 2 more
[root@artisan bin]# ^C 重试几次
[root@artisan bin]# ./mqadmin clusterlist -n 192.168.18.130:9876
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
#Cluster Name #Broker Name #BID #Addr #Version #InTPS(LOAD) #OutTPS(LOAD) #PCWait(ms) #Hour #SPACE
DefaultCluster broker?a 0 192.168.18.130:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436999.54 0.1360
DefaultCluster broker?a 1 192.168.18.130:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436999.54 0.1360
DefaultCluster broker?b 0 192.168.18.131:10911 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436999.54 0.1690
DefaultCluster broker?b 1 192.168.18.131:10811 V4_3_2 0.00(0,0ms) 0.00(0,0ms) 0 436999.54 0.1690
[root@artisan bin]#
可能是mqadmin没能正确的连接进去!网络抖动,这个跟底层netty的连接有关. 多重试几次,一般都没问题。
测试发送和消费消息
[root@artisan rocketmq-all-4.3.2-bin-release]# export NAMESRV_ADDR=192.168.18.131:9876
[root@artisan rocketmq-all-4.3.2-bin-release]#
[root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/tools.sh org.apache.rocketmq.example.quickstart.Producer
结果截个图:
测试消费者
[root@artisan rocketmq-all-4.3.2-bin-release]# sh bin/tools.sh org.apache.rocketmq.example.quickstart.Consumer
运行结果 截个图
是不是希望有个页面可以管理和查看集群信息呢? RocketMQ 提供了 RocketMQ Console ,下篇博文我们来看下如何通过拉取源码来搭建一套本地的RocketMQ Console吧
双机互为主备的配置
刚刚搭建的 是 左边的 ,生产环境更建议使用右侧的部署方式
配置的话 ,仅需要调整 broker-s.conf 中的 brokerName即可
131:
broker-s.conf
130 :
broker-s :
启动后,打开RocketMQConsole 看下,符合部署。
RocketMQ4.3.X配置参数
请移步 RocketMQ4.3.x 史上配置最全详解,没有之一
RocketMQ-初体验RocketMQ(03)_RocketMQ多机集群部署相关推荐
- RocketMQ 简单梳理 及 集群部署笔记【转】
一.RocketMQ 基础知识介绍 Apache RocketMQ是阿里开源的一款高性能.高吞吐量.队列模型的消息中间件的分布式消息中间件. 上图是一个典型的消息中间件收发消息的模型,RocketMQ ...
- Centos6下RocketMQ集群部署记录
一.RocketMQ基础知识介绍 Apache RocketMQ是阿里开源的一款高性能.高吞吐量.队列模型的消息中间件的分布式消息中间件. 上图是一个典型的消息中间件收发消息的模型,RocketMQ也 ...
- RocketMQ 简单梳理 及 集群部署笔记
一.RocketMQ 基础知识介绍 Apache RocketMQ是阿里开源的一款高性能.高吞吐量.队列模型的消息中间件的分布式消息中间件. 上图是一个典型的消息中间件收发消息的模型,RocketMQ ...
- RocketMQ初探(五)之RocketMQ4.2.6集群部署(单Master+双Master+2m+2s+async异步复制)
以下部署方式结合众多博友的博客,经过自己一步一步实际搭建,如有雷同,侵权行为,请见谅...其中遇到不少的坑,希望能帮到更多的人,现在很少能找到一份完整版4.2.6版本的搭建教程了,如果你有幸遇见,那么 ...
- RocketMQ集群部署结构
RocketMQ四大核心组成部分:NameServer.Broker.Producer以及Consumer四部分: 各组件通讯 Broker与Name Server集群中的所有节点建立长连接: Pro ...
- RocketMQ集群部署记录
RocketMQ集群部署记录 #引用 https://cloud.tencent.com/developer/article/1147765 一.RocketMQ基础知识介绍 Apache Ro ...
- (三)RocketMQ集群部署实践
2019独角兽企业重金招聘Python工程师标准>>> 全篇参照–<MyRocketMQ集群部署实战-双master-双slave-同步双写-异步刷盘(7台机器) - tant ...
- Linux服务集群部署实战--MySQL、Redis、ES、RocketMQ、Zookeeper
部署架构 部署计划 MySQL服务部署 架构 规划 部署pxc集群 部署MySQL主从架构 部署mycat集群 创建表以及测试 部署HAProxy redis集群部署 redis集群采用3主3从的架构 ...
- RocketMQ集群部署方案(DLedger)
RocketMQ集群部署方案(DLedger) 一.基本配置 1.准备三台虚拟机,root密码 root ;IP地址: 192.168.xxx.xxx worker1 192.168.xxx.xxx ...
最新文章
- openresty 安装
- 《T-SQL性能调优秘笈——基于SQL Server 2012 窗口函数》——1.7 小结
- PLSQL基础语法二-流程控制,循环
- 如何二值图转化为灰度图_木工真空吸附雕刻机如何用精雕5.21把精雕图模型转为灰度图...
- 更易型算法(Manipulating Algorithms)
- 前端也要懂Http缓存机制
- javascript通用验证
- 如何看当前windows是utf8还是gbk_监理工程师5月份出教材,现在如何备考?
- Bootstrap 固定导航条
- 程序员分析一线城市 1000 +岗位招聘需求,告诉你如何科学找工作
- Javascript基础学习12问(四)
- Umbraco中根据ID获取IPublishedContent
- Pytorch ——基础指北_零 [神经元和激活函数介绍]
- ChucK初步(10)
- Kettle连接MySQL数据库报错:Driver class ‘org.gjt.mm.mysql.Driver‘ could not be found
- 每日一犬 · 哈瓦那犬
- 空间曲面构造及其方程
- Python 视频制作神器 -- Manim入门篇
- tabIndex 和 aria注意点
- 微信jsapi支付获取code_微信JSAPI公众号支付在部分机型上出现appid参数错误的解决办法 - YangJunwei...
热门文章
- Rxjava 优雅的实现短信验证码发送
- vins中imu融合_双目版 VINS 项目发布,小觅双目摄像头作为双目惯导相机被推荐...
- TAS-LR 论文辅助笔记 图拉普拉斯正则项推导
- 文巾解题 185. 部门工资前三高的所有员工
- MATLAB从入门到精通-APP调用simulink中的参数,并且修改,将结果返回到APP中
- Python科学计算包应用-教你以可视化的方式打开NumPy
- matlab 小波变换_matlab小波工具箱实例(二):时频分析和连续小波变换
- 追源索骥:透过源码看懂Flink核心框架的执行流程
- Spring Security Architecture--官方
- Java 编程的动态性,第 7 部分: 用 BCEL 设计字节码--转载