oVirt 4.4.10三节点超融合集群安装配置及集群扩容(三)
2024-04-22 22:34:47
本篇主要记录安装及使用过程中遇到的问题<包含4.4.x, 4.5.x>
- 设置engine管理页面可以通过IP访问
- ssh连接engine服务器并在/etc/ovirt-engine/engine.conf.d新建99-custom-sso-setup.conf，添加engine节点的IP或出口IP
SSO_ALTERNATE_ENGINE_FQDNS="engine103.cluster.local 192.168.5.103 61.x.x.8"
- 重启engein服务
systemctl restart ovirt-engine
- 重启服务器后节点的9090端口服务无法访问
解决方法: 登录服务器确认ovirt-ha-agent服务是否启动成功
systemctl status ovirt-ha-agent
- 添加完集群后节点异常如下，此问题主机是集群内物理服务器硬件不一致引起的
解决方法: 查看异常节点的CPU类型，然后将集群的CPU类型修改为异常节点CPU类型，生产环境保证硬件配置一致则没有此问题
这里异常节点node212.com的CPU类型为:Secure Intel Nehalem Family
在”计算”->”集群”，选择当前集群后点击”编辑”，将CPU类型修改为Secure Intel Nehalem Family
修改完成后自动更新节点信息，以下为正常节点信息
- 访问cockpit时提示”这个浏览器太老，无法运行 Web 控制台（缺少 selector(:is():where())）”
此问题主要是更新Chrome后引起
解决方法:
sed 's/selector(:is():where())/selector(:is(*):where(*))/' -i.bak /usr/share/cockpit/static/login.js
重启cockpit服务:
systemctl restart cockpit
- 出现数据有未同步的问题
查看对应的gfs卷信息有未连接的节点
[root@node100 ~]# gluster volume heal data info
Brick node100.cluster.local:/gluster_bricks/data/data
Status: Transport endpoint is not connected
Number of entries: -Brick node101.cluster.local:/gluster_bricks/data/data
/3ebbb7ec-6334-4dff-8c84-02868c732d53/dom_md/ids
/
/.shard/.remove_me
Status: Connected
Number of entries: 3Brick node102.cluster.local:/gluster_bricks/data/data
/3ebbb7ec-6334-4dff-8c84-02868c732d53/dom_md/ids
/
/.shard/.remove_me
Status: Connected
Number of entries: 3
解决方法:
- 尝试在”存储”->”卷”下重启对应的卷
- 通过命令解决
gluster volume start data forcegluster volume heal data full
- 部署gfs时报如下错误
解决方法:
系统更新后python版本发生变化导致找不到jmespath模块，根据实际修改python版本
[root@node210 ~]# python3.11 -m ensurepip --default-pip
[root@node210 ~]# python3.11 -m pip install jmespath
- 安装gfs时报The error was: ‘str object’ has no attribute ‘vgname’
原因: ansible 2.12之前处理json语法问题
解决方法:
[root@node210 ~]# sed -i.bak 's/output | to_json/output/' /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/get_vg_groupings.yml
[root@node210 ~]# sed -i.bak 's/output | to_json/output/' /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/thick_lv_create.yml
[root@node210 ~]# sed -i.bak 's/output | to_json/output/' /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/thin_pool_create.yml
[root@node210 ~]# sed -i.bak 's/output | to_json/output/' /etc/ansible/roles/gluster.infra/roles/backend_setup/tasks/thin_volume_create.yml
- 重启3个node节点后进入emergency模式
解决方法: 按提示输入root用户密码后编辑/etc/fstab将gfs挂载配置先注释后再重启系统,重启完成后进入系统取消gfs挂载注释
访问每个node节点的后台管理启用逻辑卷并挂载gfs
点击”启动逻辑卷”，如果是第一个执行的节点则需要再点击”挂载”，当第一个节点挂载完成后后续其他的节点只需启动逻辑卷即可:
- 安装engine时报FQDN验证失败，是因检测网络时缺少netaddr库
解决方法:
[root@node210 ~]# python3.11 -m pip install netaddr
- 安装engine页面加载不出来报如下错误，未解决
- 安装完成gfs后进入engine安装界面时报如下错误，未解决
- 执行engine-setup后提示ovirt-imageio启动失败，此时在/etc/ovirt-imageio/conf.d增加如下配置开启debug
[root@ovirt454 conf.d]# cat /etc/ovirt-imageio/conf.d/99-local.conf
[logger_root]
level = DEBUG
[root@ovirt454 conf.d]# systemctl start ovirt-imageio
执行启动命令后查看/var/log/ovirt-imageio/daemon.log
如果是没有ssl证书则执行以下命令后再重新执行一次engine-setup，执行前建议先执行一次清理并重启系统
[root@ovirt454 conf.d]# engine-cleanup
[root@ovirt454 conf.d]# reboot
[root@ovirt454 conf.d]# vdsm-tool configure --module certificatesChecking configuration status...Running configure...
Reconfiguration of certificates is done.Done configuring modules to VDSM.
- 上传ISO镜像时提示注册CA证书，此问题可能和all-in-one安装模式有关
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/W5WY4K45IA6O4NA2CTALVOSPUKLHESFB/
解决方法:
- 复制vdsm配置文件
[root@ovirt454 conf.d]# cp /etc/ovirt-imageio/conf.d/60-vdsm.conf /etc/ovirt-imageio/conf.d/99-local.conf
- 修改99-local.conf中ssl证书为/etc/ovirt-imageio/conf.d/50-engine.conf中的证书
# $ cat /etc/ovirt-imageio/conf.d/99-local.conf
# [logger_root]
# level = DEBUG
#
# This example overrides ovirt-imageio service log level to DEBUG.[tls]
# TLS is required for vdsm, do not change.
enable = truekey_file = /etc/pki/ovirt-engine/keys/apache.key.nopass
cert_file = /etc/pki/ovirt-engine/certs/apache.cer
ca_file = /etc/pki/ovirt-engine/apache-ca.pem# Required vdsm private key file, do not change.
#key_file = /etc/pki/vdsm/keys/vdsmkey.pem# Required vdsm certificate file, do not change.
#cert_file = /etc/pki/vdsm/certs/vdsmcert.pem# Required vdsm CA certificate file, do not change.
#ca_file = /etc/pki/vdsm/certs/cacert.pem[remote]
# Remote service port. Changing this value requires change in the firewall
# rules on the host, and changing this value in engine configuration. vdsm
# assumes this port, don't change it.
port = 54322[local]
# Required for legacy and unsupported all-in-one setup. Improves performance
# for local clients that use the unix socket.
enable = true[control]
# Required for legacy and unsupported all-in-one setup. Vdsm depends on this,
# do not change.
transport = unix
- 重启ovirt-imageio服务
[root@ovirt454 conf.d]# systemctl restart ovirt-imageio
[root@ovirt454 conf.d]# netstat -nplt | grep 54322
tcp6 0 0 :::54322 :::* LISTEN 69337/python3
- 将54322端口放开
[root@ovirt454 conf.d]# firewall-cmd --zone=public --permanent --add-port=54322/tcp
[root@ovirt454 conf.d]# firewall-cmd --reload
[root@ovirt454 conf.d]# firewall-cmd --list-ports --zone=public
22/tcp 9986/tcp 54322/tcp 6081/udp
- 验证
- 创建虚拟机时给虚拟机设置静态IP
在"新建虚拟机"–> "初始运行"中设置虚拟机静态IP,如下所示:
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
GFS常用命令
- 查看节点状态
[root@node100 ~]# gluster peer status
Number of Peers: 2Hostname: node101.com
Uuid: 8a1adcef-323a-4f34-ae6f-e12ab7e1c02d
State: Peer in Cluster (Connected)Hostname: node102.com
Uuid: df319280-dde2-48a3-8e86-9cfff76384b8
State: Peer in Cluster (Connected)
- 查看集群节点信息
[root@node100 ~]# gluster pool list
UUID Hostname State
8a1adcef-323a-4f34-ae6f-e12ab7e1c02d node101.com Connected
df319280-dde2-48a3-8e86-9cfff76384b8 node102.com Connected
d0dc72bb-4f5a-48a3-93dc-1dbb7c0fba55 localhost Connected
- 查看集群中所有的卷
[root@node100 ~]# gluster volume list
data
engine
vmstore
- 查看某个卷的状态
[root@node100 ~]# gluster volume status data
Status of volume: data
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick node100.com:/gluster_bricks/data/data 49155 0 Y 386797
Brick node101.com:/gluster_bricks/data/data 49152 0 Y 7510
Brick node102.com:/gluster_bricks/data/data 49152 0 Y 2219
Self-heal Daemon on localhost N/A N/A Y 389738
Self-heal Daemon on node211.com N/A N/A Y 7785
Self-heal Daemon on node212.com N/A N/A Y 20415Task Status of Volume data
------------------------------------------------------------------------------
There are no active volume tasks
- 查看某个卷的信息
[root@node100 ~]# gluster volume info dataVolume Name: data
Type: Replicate
Volume ID: 18d13836-b0ba-44ba-845f-b4ab46f6a2e3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node100.com:/gluster_bricks/data/data
Brick2: node101.com:/gluster_bricks/data/data
Brick3: node102.com:/gluster_bricks/data/data
Options Reconfigured:
performance.client-io-threads: on
nfs.disable: on
transport.address-family: inet
storage.fips-mode-rchecksum: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.low-prio-threads: 32
network.remote-dio: off
performance.strict-o-direct: on
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
cluster.choose-local: off
client.event-threads: 4
server.event-threads: 4
network.ping-timeout: 30
server.tcp-user-timeout: 20
server.keepalive-time: 10
server.keepalive-interval: 2
server.keepalive-count: 5
cluster.lookup-optimize: off
storage.owner-uid: 36
storage.owner-gid: 36
cluster.granular-entry-heal: enable
- 查看集群所有卷的信息
[root@node100 ~]# gluster volume infoVolume Name: data
Type: Replicate
Volume ID: 18d13836-b0ba-44ba-845f-b4ab46f6a2e3
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node100.com:/gluster_bricks/data/data
Brick2: node101.com:/gluster_bricks/data/data
Brick3: node102.com:/gluster_bricks/data/data
Options Reconfigured:
performance.client-io-threads: on
nfs.disable: on
transport.address-family: inet
storage.fips-mode-rchecksum: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.low-prio-threads: 32
network.remote-dio: off
performance.strict-o-direct: on
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
cluster.choose-local: off
client.event-threads: 4
server.event-threads: 4
network.ping-timeout: 30
server.tcp-user-timeout: 20
server.keepalive-time: 10
server.keepalive-interval: 2
server.keepalive-count: 5
cluster.lookup-optimize: off
storage.owner-uid: 36
storage.owner-gid: 36
cluster.granular-entry-heal: enableVolume Name: engine
Type: Replicate
Volume ID: c9d72bc2-9cc5-4619-b0e9-aa095b2f066f
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node100.com:/gluster_bricks/engine/engine
Brick2: node101.com:/gluster_bricks/engine/engine
Brick3: node102.com:/gluster_bricks/engine/engine
Options Reconfigured:
performance.client-io-threads: on
nfs.disable: on
transport.address-family: inet
storage.fips-mode-rchecksum: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.low-prio-threads: 32
network.remote-dio: off
performance.strict-o-direct: on
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
cluster.choose-local: off
client.event-threads: 4
server.event-threads: 4
network.ping-timeout: 30
server.tcp-user-timeout: 20
server.keepalive-time: 10
server.keepalive-interval: 2
server.keepalive-count: 5
cluster.lookup-optimize: off
storage.owner-uid: 36
storage.owner-gid: 36
cluster.granular-entry-heal: enableVolume Name: vmstore
Type: Replicate
Volume ID: a2c90ea2-ae1c-4c2d-8558-0b3d5db17c2b
Status: Started
Snapshot Count: 0
Number of Bricks: 1 x 3 = 3
Transport-type: tcp
Bricks:
Brick1: node100.com:/gluster_bricks/vmstore/vmstore
Brick2: node101.com:/gluster_bricks/vmstore/vmstore
Brick3: node102.com:/gluster_bricks/vmstore/vmstore
Options Reconfigured:
performance.client-io-threads: on
nfs.disable: on
transport.address-family: inet
storage.fips-mode-rchecksum: on
performance.quick-read: off
performance.read-ahead: off
performance.io-cache: off
performance.low-prio-threads: 32
network.remote-dio: off
performance.strict-o-direct: on
cluster.eager-lock: enable
cluster.quorum-type: auto
cluster.server-quorum-type: server
cluster.data-self-heal-algorithm: full
cluster.locking-scheme: granular
cluster.shd-max-threads: 8
cluster.shd-wait-qlength: 10000
features.shard: on
user.cifs: off
cluster.choose-local: off
client.event-threads: 4
server.event-threads: 4
network.ping-timeout: 30
server.tcp-user-timeout: 20
server.keepalive-time: 10
server.keepalive-interval: 2
server.keepalive-count: 5
cluster.lookup-optimize: off
storage.owner-uid: 36
storage.owner-gid: 36
cluster.granular-entry-heal: enable
- 查看集群所有卷的状态
[root@node100 ~]# gluster volume status
Status of volume: data
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick node100.com:/gluster_bricks/data/data 49155 0 Y 386797
Brick node101.com:/gluster_bricks/data/data 49152 0 Y 7510
Brick node102.com:/gluster_bricks/data/data 49152 0 Y 2219
Self-heal Daemon on localhost N/A N/A Y 389738
Self-heal Daemon on node211.com N/A N/A Y 7785
Self-heal Daemon on node212.com N/A N/A Y 20415Task Status of Volume data
------------------------------------------------------------------------------
There are no active volume tasksStatus of volume: engine
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick node100.com:/gluster_bricks/engine/en
gine 49156 0 Y 389713
Brick node101.com:/gluster_bricks/engine/en
gine 49153 0 Y 7566
Brick node102.com:/gluster_bricks/engine/en
gine 49153 0 Y 2263
Self-heal Daemon on localhost N/A N/A Y 389738
Self-heal Daemon on node211.com N/A N/A Y 7785
Self-heal Daemon on node212.com N/A N/A Y 20415Task Status of Volume engine
------------------------------------------------------------------------------
There are no active volume tasksStatus of volume: vmstore
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick node100.com:/gluster_bricks/vmstore/v
mstore 49157 0 Y 386819
Brick node101.com:/gluster_bricks/vmstore/v
mstore 49154 0 Y 7655
Brick node102.com:/gluster_bricks/vmstore/v
mstore 49154 0 Y 2282
Self-heal Daemon on localhost N/A N/A Y 389738
Self-heal Daemon on node211.com N/A N/A Y 7785
Self-heal Daemon on node212.com N/A N/A Y 20415Task Status of Volume vmstore
------------------------------------------------------------------------------
There are no active volume tasks
- 启动一个卷，加force强制启动
[root@node100 ~]# gluster volume start data [force]
- 查看卷详细信息
[root@node100 ~]# gluster volume heal <volume_name> info
- 启动 GlusterFS 卷的自动修复功能
[root@node100 ~]# gluster volume heal <volume_name> full
oVirt 4.4.10三节点超融合集群安装配置及集群扩容(三)相关推荐
- 超融合服务器连接虚拟机,H3C UIS 6.5超融合产品新增虚拟机配置指导-5W100
虚拟机是由一组文件构成的一个完整系统.它具有CPU.内存.网络设备.存储设备和BIOS.虚拟机中操作系统和应用程序的运行方式与在普通物理机上的运行方式没有任何区别. 创建虚拟机是服务器虚拟化管理软件最 ...
- 【多图超详细】从零开始安装配置Cuckoo sandbox并提交样本进行分析
[多图超详细]从零开始安装配置Cuckoo sandbox并提交样本进行分析 文章目录 [多图超详细]从零开始安装配置Cuckoo sandbox并提交样本进行分析 1. 功能介绍 1.1主要功能 1 ...
- 一步步教你Hadoop多节点集群安装配置
一步步教你Hadoop多节点集群安装配置 1.集群部署介绍 1.1 Hadoop简介 Hadoop是Apache软件基金会旗下的一个开源分布式计算平台.以Hadoop分布式文件系统HDFS(Hado ...
- 书生云10亿元超融合大单的背后
2017年4月11日,书生云承建凤凰创新园EB级超融合数据中心签约暨合作伙伴大会在北京召开.凤凰创新园在市场金额方面投资达到10多亿元,这相当于中国超融合一体机市场上一年全年的市场总量,也是全球超融合 ...
- Proxmox VE(PVE)+ceph+物理网络规划-超融合生产环境安装部署案例
1 Proxmox Virtual Environment介绍 Proxmox VE 是用于企业虚拟化的开源服务器管理平台.它在单个平台上紧密集成了KVM虚拟机管理程序和LXC,软件定义的存储以及网络 ...
- FreeBSD下安装配置Hadoop集群(三)
先给自己做个广告: 开源Hive管理工具phpHiveAdmin今日更新0.05 beta2 ChangeLog: 1. sql查询页全部重写,复杂查询现在可以用异步非阻塞的方式实时的查看map/re ...
- 超详细Ubuntu Linux安装配置 Tomcat
Ubuntu 安装配置Tomcat 1. 开始之前 2. 安装 Java 2.1 查找 2.2 安装 2.3 验证 3. 安装 Tomcat 3.1 获取Tomcat 3.2 解压Tomcat 3.3 ...
- 基于SSM框架的个性化眼镜线上销售系统+论文第三稿+已降重+包安装配置
项目名称 基于SSM框架的个性化眼镜线上销售系统 视频效果 基于SSM框架的个性化眼镜线上销售系统 项目地址: https://download.csdn.net/download/m0_721809 ...
- Nagios+pnp4nagios+rrdtool 安装配置为nagios添加自定义插件(三)
nagios博大精深,可以以shell.perl等语句为nagios写插件,来满足自己监控的需要.本文写mysql中tps.qps的插件,并把收集到的结果以图形形式展现出来,这样输出的结果就有一定的要 ...
最新文章
- 《跟菜鸟学Cisco UC部署实战》-第 1 章 规划-课件(一共12章，免费)
- 深度学习难，这本书让你轻松学深度学习
- 使用WebDriver + Java + Junit做自动化测试教程
- 皮一皮：感受下什么叫瑟瑟发抖的寒冬...
- java 监听队列_spring+activemq实战之配置监听多队列实现不同队列消息消费
- Nginx 访问日志轮询切割
- Qt Creator应用3D效果
- Loadrunner中socket协议中的三个关联函数
- route 更改网络设置
- 小汤学编程之JavaScript学习day05——DOM、事件
- C++_弱引用 强引用_weak_ptr/share_ptr
- 论文查重系统如何才安全？
- Java Holder 使用
- 交住房公积金的好处及作用
- Android 使用 MediaCodec API音频播放器
- 实现PPT演示的＂非全屏播放＂效果
- 如何将网站转化为桌面应用
- 【win+linux双系统如何正确卸载linux系统】
- 亲历5.12汶川大地震----杨大伟作为志愿者14日奔赴汶川等地抗震救灾
- 盘点那些争议最大的编程观点，你是什么看法呢？