openstack rabbitmq报错导致识别不到计算节点 解决方案
2024-04-29 19:50:04
1 问题的现象
[root@controller160 ~]# less /var/log/rabbitmq/rabbit@controller160.log
2020-07-16 14:32:25.765 [error] <0.21731.1548> Channel error on connection <0.20499.1547> (172.16.1.160:34298 -> 172.16.1.160:5672, vhost: '/', user: 'rabbitmq'), channel 1:
operation queue.declare caused a channel exception not_found: failed to perform operation on queue 'conductor.controller160' in vhost '/' due to timeout
2020-07-16 14:32:25.834 [warning] <0.20499.1547> closing AMQP connection <0.20499.1547> (172.16.1.160:34298 -> 172.16.1.160:5672 - nova-conductor:156924:5a96df21-0cf1-4172-99b3-14077dc0d3f2, vhost: '/', user: 'rabbitmq'):[root@controller160 ~]# openstack network agent list
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| 12f5026f-2a62-49a2-8d87-ee0ca71688ea | Metadata agent | compute163 | None | XXX | UP | neutron-metadata-agent |
| 7014261e-6719-447e-882a-19f45033a6c9 | Metadata agent | compute164 | None | XXX | UP | neutron-metadata-agent |
| 767a9db2-7b3a-4edc-833a-39f4c3812b70 | DHCP agent | compute163 | nova | XXX | UP | neutron-dhcp-agent |
| 8df5c9ee-8086-4351-8c66-866b2d2577fa | L3 agent | compute164 | nova | XXX | UP | neutron-l3-agent |
| aa234650-cf75-4fe4-af48-0ea14495b1a6 | DHCP agent | compute164 | nova | XXX | UP | neutron-dhcp-agent |
| b7cf0b1d-4ff0-4314-9427-8cfda3419e15 | Linux bridge agent | compute164 | None | XXX | UP | neutron-linuxbridge-agent |
| c510acb9-7123-4ae3-850c-5e5184b6542a | Linux bridge agent | compute163 | None | XXX | UP | neutron-linuxbridge-agent |
| c6660e76-c613-4980-90ba-23c5585b79d5 | Open vSwitch agent | compute163 | None | XXX | UP | neutron-openvswitch-agent |
| dadea42c-e242-494a-a876-098cd71bcfc4 | L3 agent | compute163 | nova | XXX | UP | neutron-l3-agent |
| e611f91f-f162-47be-aebc-d6eaf824e0d0 | Open vSwitch agent | compute164 | None | XXX | UP | neutron-openvswitch-agent |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
[root@compute163 ~]# less /var/log/nova/nova-compute.log
2020-07-16 14:35:34.073 381825 ERROR oslo_service.service oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on controller160:5672 after inf tries: Queue.declare: (4
04) NOT_FOUND - failed to perform operation on queue 'compute.compute163' in vhost '/' due to timeout2 解决方案
#具体原因没有找出来，因此直接使用暴力的方式直接重建
#重建rabbitmq集群
#rabbitmq所在的节点都必须执行，这里只以一台为例
#停止每台节点的集群
[root@controller160 bin]# rabbitmqctl stop_app
Stopping rabbit application on node rabbit@controller160 ...
#停止rabbitmq服务
[root@controller160 bin]# systemctl stop rabbitmq-server.service
#删除或者移动mnesia目录,我这里用的是移动
[root@controller160 mnesia]# ll
total 8
drwxr-x--- 4 rabbitmq rabbitmq 4096 Jul 16 16:24 rabbit@controller160
-rw-r----- 1 rabbitmq rabbitmq 64 Jun 18 00:20 rabbit@controller160-feature_flags
drwxr-x--- 8 rabbitmq rabbitmq 178 Jul 16 15:51 rabbit@controller160-plugins-expand
[root@controller160 mnesia]# pwd
/var/lib/rabbitmq/mnesia
[root@controller160 ~]# mv /var/lib/rabbitmq/mnesia/ /tmp/mnesia#任选1个控制节点首先启动rabbitmq服务，这里选择controller160节点
[root@controller160 ~]#systemctl start rabbitmq-server.service
[root@controller160 ~]#rabbitmqctl cluster_status#分发.erlang.cookie
[root@controller160 ~]#scp /var/lib/rabbitmq/.erlang.cookie root@172.16.1.161:/var/lib/rabbitmq/
[root@controller160 ~]#scp /var/lib/rabbitmq/.erlang.cookie root@172.16.1.162:/var/lib/rabbitmq/#修改controller161/162节点.erlang.cookie文件的用户/组，以controller161节点为例
[root@controller161 ~]#chown rabbitmq:rabbitmq /var/lib/rabbitmq/.erlang.cookie#注意修改全部控制节点.erlang.cookie文件的权限，默认即400权限，可不修改
[root@controller161 ~]#ll /var/lib/rabbitmq/.erlang.cookie#启动controller161/162节点的rabbitmq服务
[root@controller161 ~]#systemctl restart rabbitmq-server[root@controller162 ~]#systemctl restart rabbitmq-server#构建集群，controller161/162节点以ram节点的形式加入集群
[root@controller161 ~]#rabbitmqctl stop_app
[root@controller161 ~]#rabbitmqctl join_cluster --ram rabbit@controller160
[root@controller161 ~]#rabbitmqctl start_app#重新授权，确认都为管理员权限
rabbitmqctl set_user_tags rabbitmq administrator
rabbitmqctl set_permissions -p "/" rabbitmq ".*" ".*" ".*"
[root@controller160 ~]# rabbitmqctl list_users
Listing users ...
user tags
rabbitmq [administrator]
guest [administrator]#最后依次重启每个节点的rabbitmq
systemctl restart rabbitmq-server
3 验证服务是否恢复：
#查看日志，已无报错
[root@controller160 ~]# openstack network agent list
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| ID | Agent Type | Host | Availability Zone | Alive | State | Binary |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
| 12f5026f-2a62-49a2-8d87-ee0ca71688ea | Metadata agent | compute163 | None | :-) | UP | neutron-metadata-agent |
| 7014261e-6719-447e-882a-19f45033a6c9 | Metadata agent | compute164 | None | :-) | UP | neutron-metadata-agent |
| 767a9db2-7b3a-4edc-833a-39f4c3812b70 | DHCP agent | compute163 | nova | :-) | UP | neutron-dhcp-agent |
| 8df5c9ee-8086-4351-8c66-866b2d2577fa | L3 agent | compute164 | nova | :-) | UP | neutron-l3-agent |
| aa234650-cf75-4fe4-af48-0ea14495b1a6 | DHCP agent | compute164 | nova | :-) | UP | neutron-dhcp-agent |
| b7cf0b1d-4ff0-4314-9427-8cfda3419e15 | Linux bridge agent | compute164 | None | XXX | UP | neutron-linuxbridge-agent |
| c510acb9-7123-4ae3-850c-5e5184b6542a | Linux bridge agent | compute163 | None | XXX | UP | neutron-linuxbridge-agent |
| c6660e76-c613-4980-90ba-23c5585b79d5 | Open vSwitch agent | compute163 | None | :-) | UP | neutron-openvswitch-agent |
| dadea42c-e242-494a-a876-098cd71bcfc4 | L3 agent | compute163 | nova | :-) | UP | neutron-l3-agent |
| e611f91f-f162-47be-aebc-d6eaf824e0d0 | Open vSwitch agent | compute164 | None | :-) | UP | neutron-openvswitch-agent |
+--------------------------------------+--------------------+------------+-------------------+-------+-------+---------------------------+
[root@controller160 ~]# openstack compute service list
+----+----------------+---------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+----+----------------+---------------+----------+---------+-------+----------------------------+
| 13 | nova-conductor | controller160 | internal | enabled | up | 2020-07-16T15:59:13.000000 |
| 19 | nova-scheduler | controller160 | internal | enabled | up | 2020-07-16T15:59:08.000000 |
| 20 | nova-conductor | controller162 | internal | enabled | up | 2020-07-16T15:59:01.000000 |
| 23 | nova-scheduler | controller162 | internal | enabled | up | 2020-07-16T15:58:43.000000 |
| 25 | nova-conductor | controller161 | internal | enabled | up | 2020-07-16T15:59:18.000000 |
| 26 | nova-scheduler | controller161 | internal | enabled | up | 2020-07-16T15:59:17.000000 |
| 47 | nova-compute | compute163 | nova | enabled | up | 2020-07-16T15:59:27.000000 |
| 48 | nova-compute | compute164 | nova | enabled | up | 2020-07-16T15:59:27.000000 |
+----+----------------+---------------+----------+---------+-------+----------------------------+openstack rabbitmq报错导致识别不到计算节点 解决方案相关推荐
- linux 启动rabbitmq 报错：
linux 启动rabbitmq 报错: Job for rabbitmq-server.service failed because the control process exited with ...
- Windows下安装RabbitMQ报错：unable to perform an operation on node时的解决方案
Windows下安装RabbitMQ报错:unable to perform an operation on node时的解决方案 参考文章: (1)Windows下安装RabbitMQ报错:unab ...
- rabbitmq报错：Listener method could not be invoked with the incoming message
rabbitmq报错:Listener method could not be invoked with the incoming message 错误翻译:无法使用传入消息调用监听器方法 一个top ...
- Oracle EBS 销售订单发运后连接行程停靠站请求报错导致物料未出库，但单销售行已关闭并进入开票接口
Oracle EBS 销售订单发运后连接行程停靠站请求报错导致物料未出库,但单销售行已关闭并进入开票接口 在开发销售订单发运的程序时碰到如图所示的错误: 在分析一波可能出现的问题后定位到销售订单类型中 ...
- 安装rabbitMq报错:error: unpacking of archive failed on file /usr/lib/rabbitmq/lib/rabbitmq_server-3.8.9
安装rabbitMq报错: 错误提示:error: unpagacking of archive failed on file /usr/lib/rabbitmq/lib/rabbitmq_serve ...
- Rabbitmq报错pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)
rabbitmq 报错 pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',) ...
- ANSYS-CFX，计算时报错，内存参数报错，return code 1【终极解决方案】
ANSYS-CFX,计算时报错,内存参数报错,return code 1[终极解决方案] 在CFX计算时经常会遇到内存不足的错误报告,有的算例网格并不多也会出现这样的问题,本文就最近遇到的内存错误问题 ...
- VMware虚拟机更改静态IP报错Unit network.service entered failed state解决方案
VMware虚拟机更改静态IP报错Unit network.service entered failed state解决方案 问题背景 解决方案一 解决方案二 Lyric: 就算没伴奏也可以继续 问题 ...
- SQLSERVER 创建ODBC 报错的解决办法 SQLState:‘01000‘的解决方案
SQLSERVER 创建ODBC 报错的解决办法 SQLState:'01000'的解决方案 参考文章: (1)SQLSERVER 创建ODBC 报错的解决办法 SQLState:'01000'的解决 ...
- Linux下安装Docker，报错docker: unrecognized service的两种解决方案
Linux下安装Docker,报错docker: unrecognized service的两种解决方案 参考文章: (1)Linux下安装Docker,报错docker: unrecognized ...
最新文章
- Opencv卷积滤波cvFilter2D-高通与低通
- 计算机组成原理——概述
- cocos2d-x游戏实例（15）-纵版射击游戏（2）
- itext替换pdf中的中文
- file_get_contents js没有渲染数据_浏览器渲染页面那些事
- mysql plus baomidou_com.baomidou.mybatisplus.core.mapper 不存在
- windows执行命令来运行loadrunner录制好的脚本（收藏）
- volatility 基本用法
- 测试案例：如何测试一间教室？
- 风电远程监控系统解决方案，双碳战略下新能源运维市场正当红
- Learning to Memorize Entailment and Discourse Relations for Persona-Consistent Dialogues论文学习
- 东软睿驰标准化域控制器产品正式发布
- 系统集成项目管理之项目质量管理
- 临界资源的同步与互斥，区分临界资源与临界区，二义性分析
- iOS CAAnimation 详解
- asp 支付宝 企业版 接口 支持网银接口 ，网银直接支付
- Leetcode:62题 不同路径（一个机器人位于一个 m x n 网格的左上角 。 机器人每次只能向下或者向右移动一步。机器人试图达到网格的右下角）
- 小鲁客栈---预告篇
- 想成为“牛人”的你，知道这些“壁垒”吗？——论技术壁垒、时间壁垒和成果壁垒
- 移动端兼容问题总结（摘抄总和）