postgresql 高可用 pacemaker + corosync 之二 setup vip-mas ,vip-sla 均绑定在 master
2024-05-13 02:09:59
os: ubuntu 16.04
db: postgresql 9.6.8
pacemaker: Pacemaker 1.1.14 Written by Andrew Beekhof
corosync: Corosync Cluster Engine, version ‘2.3.5’
pacemaker 集群资源管理器（Cluster Resource Management）
corosync 集群消息事务层（Massage Layer）
pcs CRM的管理接口工具
ip 规划
vip-mas 192.168.56.119 # for read write
vip-sla 192.168.56.120 # for stream replicationnode1 192.168.56.92
node2 192.168.56.90
node3 192.168.56.88其中 vip-mas 是对外提供服务的ip地址,vip-sla 是用于 stream replication 的ip地址.都同时绑定在 master 角色的主机网卡上.
生产环境时 服务网卡 和 复制网卡 应该是两个独立的物理网卡,以免相互干扰.
os 设置
# iptables -F# systemctl stop ufw;
systemctl disable ufw;禁用selinux,有的话就修改,没有就不修改(依赖policycoreutils)
# vi /etc/selinux/config SELINUX=disabled# vi /etc/hosts192.168.56.119 vip-mas # for read write
192.168.56.120 vip-sla # for stream replication192.168.56.92 node1
192.168.56.90 node2
192.168.56.88 node3postgresql stream replication
安装配置好 1 master 2 slave async stream replication.
详细过程可以参考另外的blog,注意需要禁止 postgresql 随机启动,用 pacemaker + corosync 来管理 postgresql.
# systemctl stop postgresql;
systemctl disable postgresql;安装 pacemaker corosync pcs
留意 2224 端口的使用情况
# netstat -lntp |grep -i 2224所有节点都需要安装对应的软件
# apt list |grep -Ei "pacemaker|corosync|corosync-dev|pcs|psmisc|fence-agents"# apt install -y pacemaker corosync corosync-dev pcs psmisc fence-agents crmsh# dpkg -l |grep -Ei "pacemaker|corosync|pcs|psmisc|fence-agents|crmsh"ii corosync 2.3.5-3ubuntu2.1 amd64 cluster engine daemon and utilities
ii corosync-dev 2.3.5-3ubuntu2.1 all cluster engine generic development (transitional package)
ii crmsh 2.2.0-1 amd64 CRM shell for the pacemaker cluster manager
ii fence-agents 4.0.22-2 amd64 Fence Agents for Red Hat Cluster
ii libcorosync-common-dev:amd64 2.3.5-3ubuntu2.1 amd64 cluster engine common development
ii libcorosync-common4:amd64 2.3.5-3ubuntu2.1 amd64 cluster engine common library
ii pacemaker 1.1.14-2ubuntu1.4 amd64 cluster resource manager
ii pacemaker-cli-utils 1.1.14-2ubuntu1.4 amd64 cluster resource manager command line utilities
ii pacemaker-common 1.1.14-2ubuntu1.4 all cluster resource manager common files
ii pacemaker-resource-agents 1.1.14-2ubuntu1.4 all cluster resource manager general resource agents
ii pcs 0.9.149-1ubuntu1.1 amd64 Pacemaker Configuration System
ii psmisc 22.21-2.1build1 amd64 utilities that use the proc file system对应的完全卸载指令
# apt-get -y remove --purge corosync corosync-dev libcorosync-common-dev libcorosync-common4 pacemaker pacemaker-cli-utils pacemaker-common pacemaker-resource-agents pcs psmisc fence-agents crmsh修改 hacluster 用户密码
# echo haclusterhacluster | passwd hacluster --stdin启动 pacemaker corosync pcs
在各个节点上确保三个服务的启用及自启动状态,一般情况下,初次安装完后就是启动的状态
# systemctl status pacemaker corosync pcsd;
systemctl enable pacemaker corosync pcsd;
systemctl restart pacemaker corosync pcsd;# ls -l /lib/systemd/system/corosync.service;
ls -l /lib/systemd/system/pacemaker.service;
ls -l /lib/systemd/system/pcsd.service;设置 corosync
所有节点都需要设置,注意修改 totem => interface => member => memberaddr
# cp /etc/corosync/corosync.conf /etc/corosync/corosync.conf.bak
# cat /dev/null > /etc/corosync/corosync.conf# vi /etc/corosync/corosync.conf#节点间进行心跳传播的协议，ring 0代表不需要向任何信息就能到达
totem {version: 2cluster_name: pgclustertoken: 3000token_retransmits_before_loss_const: 10clear_node_high_bit: yessecauth: offcrypto_cipher: nonecrypto_hash: noneinterface {ringnumber: 0bindnetaddr: 192.168.56.0broadcast: yes#mcastaddr: 239.255.1.1mcastport: 5405ttl: 1}
}
nodelist {node {ring0_addr: 192.168.56.92name: node1nodeid: 1}node {ring0_addr: 192.168.56.90name: node2nodeid: 2}node {ring0_addr: 192.168.56.88name: node3nodeid: 3}
}
quorum {provider: corosync_votequorumexpected_votes: 2
}
aisexec {user: rootgroup: root
}
service {name: pacemakerver: 0
}
logging {to_logfile: yeslogfile: /var/log/corosync/corosync.logto_syslog: yestimestamp: onlogger_subsys {subsys: QUORUMdebug: off}
}所有节点都重启下
# systemctl restart corosync设置 pacemaker
所有节点都需要设置
# rm -f /var/lib/pacemaker/cib/cib*所有节点都重启下 pacemaker
# systemctl restart pacemaker pcsd在 node1 节点上检查状态
# crm_mon -Afr -1Last updated: Mon Feb 18 16:11:03 2019 Last change: Mon Feb 18 16:09:32 2019 by hacluster via crmd on node3
Stack: corosync
Current DC: node3 (version 1.1.14-70404b0) - partition with quorum
3 nodes and 0 resources configuredOnline: [ node1 node2 node3 ]Full list of resources:Node Attributes:
* Node node1:
* Node node2:
* Node node3:Migration Summary:
* Node node1:
* Node node3:
* Node node2:集群认证
node1 节点上执行
# pcs cluster auth 192.168.56.92 192.168.56.90 192.168.56.88192.168.56.88: Authorized
192.168.56.90: Authorized
192.168.56.92: AuthorizedUsername 输入 hacluster
Password 输入 haclusterhacluster
或者 添加 -u -p
# pcs cluster auth -u hacluster -p haclusterhacluster 192.168.56.92 192.168.56.90 192.168.56.88查看 tokens
# cat /var/lib/pcsd/tokens{"format_version": 2,"data_version": 1,"tokens": {"192.168.56.88": "17148b7b-634c-4ffc-a013-89cf0727fa1d","192.168.56.90": "8869aa0f-56a8-4e72-9746-474df8bef48e","192.168.56.92": "a03d15fe-c9d5-4845-b16e-23b418f4c0b3"}
}# cat /var/lib/pcsd/pcs_users.conf[{"username": "hacluster","token": "a0120285-97d3-4388-a8be-bf2fffefff67","creation_date": "2019-02-18 16:12:11 +0800"}
]创建资源配置文件
node1 节点上创建 /root/cluster.pcs 文件,内容如下
内容根据具体的要求有差异
pcs cluster cib pgsql_cfgpcs -f pgsql_cfg property set no-quorum-policy="ignore"
pcs -f pgsql_cfg property set stonith-enabled="false"
pcs -f pgsql_cfg resource defaults resource-stickiness="INFINITY"
pcs -f pgsql_cfg resource defaults migration-threshold="1"pcs -f pgsql_cfg resource create vip-mas IPaddr2 \ip="192.168.56.119" \nic="eno1" \cidr_netmask="24" \op start timeout="60s" interval="0s" on-fail="restart" \op monitor timeout="60s" interval="10s" on-fail="restart" \op stop timeout="60s" interval="0s" on-fail="block"pcs -f pgsql_cfg resource create vip-sla IPaddr2 \ip="192.168.56.120" \nic="eno1" \cidr_netmask="24" \meta migration-threshold="0" \op start timeout="60s" interval="0s" on-fail="stop" \op monitor timeout="60s" interval="10s" on-fail="restart" \op stop timeout="60s" interval="0s" on-fail="ignore"pcs -f pgsql_cfg resource create pgsql pgsql \pgctl="/usr/lib/postgresql/9.6/bin/pg_ctl" \psql="/usr/lib/postgresql/9.6/bin/psql" \pgdata="/data/pg9.6/main/" \config="/etc/postgresql/9.6/main/postgresql.conf" \socketdir="/var/run/postgresql" \rep_mode="async" \node_list="node1 node2 node3" \master_ip="192.168.56.120" \repuser="repl" \primary_conninfo_opt="password=replrepl keepalives_idle=60 keepalives_interval=5 keepalives_count=5" \restart_on_promote='true' \op start timeout="60s" interval="0s" on-fail="restart" \op monitor timeout="60s" interval="4s" on-fail="restart" \op monitor timeout="60s" interval="3s" on-fail="restart" role="Master" \op promote timeout="60s" interval="0s" on-fail="restart" \op demote timeout="60s" interval="0s" on-fail="stop" \op stop timeout="60s" interval="0s" on-fail="block" \op notify timeout="60s" interval="0s"pcs -f pgsql_cfg resource master msPostgresql pgsql \master-max=1 master-node-max=1 clone-max=5 clone-node-max=1 notify=truepcs -f pgsql_cfg resource group add master-group vip-mas vip-slapcs -f pgsql_cfg constraint colocation add master-group with master msPostgresql INFINITY
pcs -f pgsql_cfg constraint order promote msPostgresql then start master-group symmetrical=false score=INFINITY
pcs -f pgsql_cfg constraint order demote msPostgresql then stop master-group symmetrical=false score=0pcs cluster cib-push pgsql_cfg加载配置文件
# sh /root/cluster.pcsAdding msPostgresql master-group (score: INFINITY) (Options: first-action=promote then-action=start symmetrical=false)
Adding msPostgresql master-group (score: 0) (Options: first-action=demote then-action=stop symmetrical=false)
CIB updatednode1 节点先执行,node2,node3 依次重启相关服务
# systemctl restart corosync pacemaker pcsd检查是否创建成功
# crm_mon -Afr -1Last updated: Mon Feb 18 16:16:37 2019 Last change: Mon Feb 18 16:16:03 2019 by root via crm_attribute on node1
Stack: corosync
Current DC: node1 (version 1.1.14-70404b0) - partition with quorum
3 nodes and 7 resources configuredOnline: [ node1 node2 node3 ]Full list of resources:Master/Slave Set: msPostgresql [pgsql]Masters: [ node1 ]Slaves: [ node2 node3 ]Resource Group: master-groupvip-mas (ocf::heartbeat:IPaddr2): Started node1vip-sla (ocf::heartbeat:IPaddr2): Started node1Node Attributes:
* Node node1:+ master-pgsql : 1000 + pgsql-data-status : LATEST + pgsql-master-baseline : 0000000006000098+ pgsql-status : PRI
* Node node2:+ master-pgsql : -INFINITY + pgsql-data-status : STREAMING|ASYNC+ pgsql-status : HS:async
* Node node3:+ master-pgsql : -INFINITY + pgsql-data-status : STREAMING|ASYNC+ pgsql-status : HS:async Migration Summary:
* Node node1:
* Node node3:
* Node node2:启动
node1 节点上启动 cluster
# pcs cluster start --all192.168.56.88: Starting Cluster...
192.168.56.90: Starting Cluster...
192.168.56.92: Starting Cluster...# pcs cluster enable --all192.168.56.92: Cluster Enabled
192.168.56.90: Cluster Enabled
192.168.56.88: Cluster Enabled检查 cluster 状态
# pcs cluster statusCluster Status:Last updated: Mon Feb 18 16:17:27 2019 Last change: Mon Feb 18 16:16:03 2019 by root via crm_attribute on node1Stack: corosyncCurrent DC: node1 (version 1.1.14-70404b0) - partition with quorum3 nodes and 7 resources configuredOnline: [ node1 node2 node3 ]PCSD Status:node1 (192.168.56.92): Onlinenode2 (192.168.56.90): Onlinenode3 (192.168.56.88): Online检查 corosync
# pcs status corosyncMembership information
----------------------Nodeid Votes Name3 1 192.168.56.882 1 192.168.56.901 1 192.168.56.92 (local)检查 pacemaker
# pcs statusCluster name: pgcluster
WARNING: corosync and pacemaker node names do not match (IPs used in setup?)
Last updated: Mon Feb 18 16:17:53 2019 Last change: Mon Feb 18 16:16:03 2019 by root via crm_attribute on node1
Stack: corosync
Current DC: node1 (version 1.1.14-70404b0) - partition with quorum
3 nodes and 7 resources configuredOnline: [ node1 node2 node3 ]Full list of resources:Master/Slave Set: msPostgresql [pgsql]Masters: [ node1 ]Slaves: [ node2 node3 ]Resource Group: master-groupvip-mas (ocf::heartbeat:IPaddr2): Started node1vip-sla (ocf::heartbeat:IPaddr2): Started node1PCSD Status:node1 (192.168.56.92): Onlinenode2 (192.168.56.90): Onlinenode3 (192.168.56.88): OnlineDaemon Status:corosync: active/enabledpacemaker: active/enabledpcsd: active/enabled查看 master 节点的ip地址
root@node1:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00inet 127.0.0.1/8 scope host lovalid_lft forever preferred_lft foreverinet6 ::1/128 scope host valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000link/ether 08:00:27:f6:4a:79 brd ff:ff:ff:ff:ff:ffinet 10.0.2.101/24 brd 10.0.2.255 scope global enp0s3valid_lft forever preferred_lft foreverinet6 fe80::a00:27ff:fef6:4a79/64 scope link valid_lft forever preferred_lft forever
3: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000link/ether 08:00:27:c4:82:0d brd ff:ff:ff:ff:ff:ffinet 192.168.56.92/24 brd 192.168.56.255 scope global eno1valid_lft forever preferred_lft foreverinet 192.168.56.119/24 brd 192.168.56.255 scope global secondary eno1valid_lft forever preferred_lft foreverinet 192.168.56.120/24 brd 192.168.56.255 scope global secondary eno1valid_lft forever preferred_lft foreverinet6 fe80::a00:27ff:fec4:820d/64 scope link valid_lft forever preferred_lft forever192.168.56.119 和 192.168.56.120 已经绑定在 node1 节点的网卡上了
最后关闭 postgresql 的自启动
# systemctl disable postgresql-9.6.service;
systemctl enable corosync;
systemctl enable pacemaker;
systemctl enable pcsd;参考:
https://www.clusterlabs.org/
http://clusterlabs.org/pacemaker/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/index.html
https://wiki.clusterlabs.org/wiki/Pacemaker
https://wiki.clusterlabs.org/wiki/PgSQL_Replicated_Cluster
https://ubuntuforums.org/showthread.php?t=2329725
postgresql 高可用 pacemaker + corosync 之二 setup vip-mas ,vip-sla 均绑定在 master相关推荐
- postgresql 高可用 etcd + patroni 之二 patroni
os: centos 7.4 postgresql: 9.6.9 etcd: 3.2.18 patroni: 1.4.4 patroni + etcd 是在一个postgrsql 开源大会上 亚信的一 ...
- postgresql 高可用 patroni + etcd 之二 patroni
os: centos 7.4 postgresql: 9.6.9 etcd: 3.2.18 patroni: 1.4.4 patroni + etcd 是postgrsql 的一个高可用方案,依然是基 ...
- PostgreSQL高可用（二）使用pgpool实现主从切换+读写分离
文章目录 1主从流复制 1.1 主机分配 2主从切换 2.1 手动切换 2.2 pgpool 自动切换 2.2.1 解压编译安装 2.2.2 添加环境变量 2.2.3 配置pool_hba.conf ...
- PostgreSQL高可用（三）解决pgpool的单点问题
文章目录 主机分配 PGPOOL高可用,解决单点问题 1 图例 2 看门狗简介 3 启动/停止看门狗 4 配置看门狗 (pgpool.conf) 4.1 基本配置 4.1.1 启用 4.1.2到前端服 ...
- postgresql高可用_Postgresql高可用实现方案
本文主要讲解一种Postgresql高可用实现方案.由于项目需要,我们需要在短时间实现底层Postgresql DB的高可用,而网络上大多数的解决方案比较复杂,需要深入了解Postgresql. 背景 ...
- PostgreSQL高可用集群在360的落地实战
本文主要从以下几个方面介绍PostgreSQL高可用集群在360的落地实战 为什么选择Patroni + Etcd + PostgreSQL高可用集群方案 PostgreSQL高可用集群在360的落地 ...
- 高可用服务设计之二：Rate limiting 限流与降级
<高可用服务设计之二:Rate limiting 限流与降级> <nginx限制请求之一:(ngx_http_limit_conn_module)模块> <nginx限制 ...
- POSTGRESQL 高可用 Patroni VS Repmgr 到底哪家强（1）
在众多postgresql 高可用模式中,主要的参与者有两位, Patroni VS repmgr 基于这二者的功能优点以及缺点相信大部分人都不是太明确,下面将根据两篇翻译的文字合并,来对两个高可用的 ...
- corosync+pacemaker实现高可用(HA)集群(二)
部署方案二(推荐):corosync+pacemaker 利用ansible自动安装corosync和pacemaker 注:关于ansible的具体使用可参见"ansible实现自动化自动 ...
最新文章
- pendo android,Pendo
- 全球只有4个人通关的游戏！不过如此
- CocoaPods安装及使用《转》
- 配置ip yum 单用户 救援 运行级别
- 软件网络协议测试,网络协议测试方法是什么
- 堪比JMeter的.Net压测工具 - Crank 入门篇
- 下面是一个python函数、哪个说法不正确_2017秋python语言程序设计（北京信息科技大学）答案...
- Miniflter中 NPInstanceSetup调查
- Bean在Spring和SpringMVC中无所不在
- LTE学习笔记：LTE总体架构
- 2022年全国图书参考联盟读秀5.0/4.0/3.0/2.0/1.0书库网盘数据索引在线搜索查询系统搭建教程，可以实现ISBN/SS号/书封面链接/书名/作者/出版社…等信息一键搜索查询
- word2vec训练维基百科中文词向量
- 关于namecheap 域名运营商,域名赎回详细步骤
- T32使用-----抓取rpm dump
- “绿水青山就是金山银山”
- 基于PHP+MySQL的服装购物商城系统#毕业设计
- KONG网关和KONGA界面的入门使用，快速上手
- #今日论文推荐# 速递：机器学习帮助构建人类肠道微生物群体感应网络
- 为什么flyme不支持谷歌服务器,大家的flyme能正常使用谷歌全家桶吗
- 景区无线wifi覆盖解决方案
热门文章
- 一文读懂知识图谱的商业应用进程及技术背景
- 2021-01-19（学堂云）文献管理与信息分析期末考卷
- python培训班-天津python培训机构多少钱
- IDEA导出jar包并使用exe4j制作可执行文件
- python 移动平均线_Python 计算EMA(指数移动平均线)
- Workflow JBPM 工作流
- pxelinux.0：winboot：网络引导（启动）wim格式的windows PE系统：配置文件写法
- macOS下载的文件执行出现operation not permitted问题
- Android提示音大全,个性提示音下载-个性提示音app安卓版下载v1.2.1-游迅网
- 北京市昌平区申通快递 电话