前文环境中完成k8s单master节点集群架构部署，两个node节点都已加入集群中，完成与master节点的通信。本文在前文基础下，再加入一个master节点，同时加入nginx负载均衡+keepalived高可用集群，前文链接地址：Kubernetes------二进制单master节点集群部署

文章目录

一、K8S双master节点集群架构
二、资源分配
三、master2节点部署
四、nginx服务器部署
四、测试

一、K8S双master节点集群架构

二、资源分配

主机	IP地址	所需组件
master1	192.168.88.10	kube-apiserver、kube-controller-manager、kube-scheduler、etcd
node1	192.168.88.20	kubelet、kube-proxy、docker-ce、flannel、etcd
node2	192.168.88.30	kubelet、kube-proxy、docker-ce、flannel、etcd
master2	192.168.88.40	kube-apiserver、kube-controller-manager、kube-schedule
nginx1	192.168.88.50	nginx、keepalived
nginx2	192.168.88.60	nginx、keepalived

三、master2节点部署

所有节点修改主机名，关闭防护墙及系统安全防护机制，安全性功能，网络管理

#以master2节点为例
[root@localhost ~]# hostnamectl set-hostname master2
[root@localhost ~]# su -
上一次登录：五 10月 15 14:20:19 CST 2021从 192.168.88.1pts/1 上
[root@master2 ~]# systemctl stop firewalld.service
[root@master2 ~]# systemctl disable firewalld.service
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@master2 ~]# setenforce 0
[root@master2 ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
[root@master2 ~]# systemctl stop NetworkManager
[root@master2 ~]# systemctl disable NetworkManager
Removed symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service.
Removed symlink /etc/systemd/system/dbus-org.freedesktop.NetworkManager.service.
Removed symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service.

把master1节点相关命令，证书，服务复制到master2节点

[root@master1 ~]# scp -r /opt/kubernetes/ root@192.168.88.40:/opt[root@master1 ~]# scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.88.40:/usr/lib/systemd/system/

修改配置文件kube-apiserver

[root@master2 ~]# cd /opt/kubernetes/
[root@master2 /opt/kubernetes]# ls
bin  cfg  ssl
[root@master2 /opt/kubernetes]# cd cfg/
[root@master2 /opt/kubernetes/cfg]# ls
kube-apiserver  kube-controller-manager  kube-scheduler  token.csv
[root@master2 /opt/kubernetes/cfg]# vim kube-apiserver

把master1节点etcd证书复制到master2节点

#新加入的master节点也包含apiserver，apiserver工作时，也需要与ETCD交互，需要ETCD证书认证
[root@master1 ~]# scp -r /opt/etcd/ root@192.168.88.40:/opt/

master2节点开启相关服务（接口，调度器，控制器），并查看服务状态

设置环境变量

[root@master2 ~]# echo "export PATH=$PATH:/opt/kubernetes/bin/" >> /etc/profile
[root@master2 ~]# tail -2 /etc/profileexport PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/opt/kubernetes/bin/
[root@master2 ~]# source /etc/profile#查看集群节点信息
[root@master2 ~]# kubectl get nodes
NAME            STATUS   ROLES    AGE     VERSION
192.168.88.20   Ready    <none>   4h3m    v1.12.3
192.168.88.30   Ready    <none>   3h47m   v1.12.3

四、nginx服务器部署

清空防火墙列表

#以nginx1为例
[root@nginx1 ~]# iptables -F

设置nginx官方源，并安装服务

#设置nginx官方源，用于安装服务
[root@nginx1 ~]# vim /etc/yum.repos.d/nginx.repo
[root@nginx1 ~]# cat /etc/yum.repos.d/nginx.repo
[nginx]
name=nginx repo
baseurl=http://nginx.org/packages/centos/7/$basearch/
gpgcheck=0#重载yum仓库
[root@nginx1 ~]# yum list #安装nginx服务
[root@nginx1 ~]# yum -y install nginx

在配置文件中添加四层转发功能（使用stream模块做四层负载）

stream {log_format  main  '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';access_log  /var/log/nginx/k8s-access.log  main;upstream k8s-apiserver {server 192.168.88.10:6443;server 192.168.88.40:6443;}server {listen 6443;proxy_pass k8s-apiserver;}}

开启nginx服务

#检查语法是否正确
[root@nginx1 ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful#开启服务，查看端口
[root@nginx1 ~]# systemctl start nginx
[root@nginx1 ~]# systemctl enable nginx
Created symlink from /etc/systemd/system/multi-user.target.wants/nginx.service to /usr/lib/systemd/system/nginx.service.
[root@nginx1 ~]# netstat -natp | grep nginx
tcp        0      0 0.0.0.0:6443            0.0.0.0:*               LISTEN      5699/nginx: master
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      5699/nginx: master

下载keepalived服务

[root@nginx1 ~]# yum -y install keepalived

nginx1服务器keepalived配置

[root@nginx1 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived
global_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc} notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1smtp_connect_timeout 30router_id NGINX_MASTER
}
vrrp_script check_nginx {       script "/etc/nginx/check_nginx.sh"
}
vrrp_instance VI_1 {state MASTERinterface ens33virtual_router_id 51priority 100  advert_int 1   authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.88.100/24}track_script {      check_nginx             }
}

nginx2服务器keepalived配置

[root@nginx2 ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived
global_defs {  notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc} notification_email_from Alexandre.Cassen@firewall.locsmtp_server 127.0.0.1smtp_connect_timeout 30router_id NGINX_MASTER
}
vrrp_script check_nginx {       script "/etc/nginx/check_nginx.sh"
}
vrrp_instance VI_1 {state BACKUP              #修改为BACKUP    interface ens33virtual_router_id 51priority 90               #优先级比MASTER小，改为90advert_int 1   authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.88.100/24}track_script {      check_nginx             }
}

创建nginx检测脚本

[root@nginx1 ~]# vim /etc/nginx/check_nginx.sh
count=$(ps -ef |grep nginx |egrep -cv "grep|$$")  #过滤nginx进程数量 #如果nginx停止，keepalived也停止。
if [ "$count" -eq 0 ];thensystemctl stop keepalived
fi#赋予执行权限
[root@nginx1 /etc/nginx]# chmod +x check_nginx.sh

开启keepalived服务

[root@nginx1 ~]# systemctl start keepalived.service

nginx1服务器查看是否有飘逸地址

nginx1服务器关闭nginx服务，nginx2服务器查看飘逸地址是否转移

nginx1服务器重新开启nginx和keepalived服务，查看漂移地址是否回来

修改node节点配置文件（bootstrap.kubeconfig、kubelet.kubeconfig、kube-proxy.kubeconfig），所有server地址统一指向VIP（192.168.88.100）

vim /opt/kubernetes/cfg/bootstrap.kubeconfig
vim /opt/kubernetes/cfg/kubelet.kubeconfig
vim /opt/kubernetes/cfg/kube-proxy.kubeconfig #检查
[root@node1 ~]# cd /opt/kubernetes/cfg/
[root@node1 /opt/kubernetes/cfg]# grep 100 *
bootstrap.kubeconfig:    server: https://192.168.88.100:6443
kubelet.kubeconfig:    server: https://192.168.88.100:6443
kube-proxy.kubeconfig:    server: https://192.168.88.100:6443

四、测试

master1节点创建pod

#run在群集中运行一个指定镜像  nginx容器名称  --image=nginx镜像名称
[root@master1 ~]# kubectl run nginx --image=nginx
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx created#查看pod
[root@master1 ~]# kubectl get pods
NAME                    READY   STATUS              RESTARTS   AGE
nginx-dbddb74b8-hkn7h   0/1     ContainerCreating   0          19s#容器有创建过程，创建过程中状态显示ContainerCreating，完成后显示Running#再次查看
[root@master1 ~]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-hkn7h   1/1     Running   0          104s

查看pod所在节点（详细信息）

[root@master1 ~]# kubectl get pods -o wide
NAME                    READY   STATUS    RESTARTS   AGE     IP            NODE            NOMINATED NODE
nginx-dbddb74b8-hkn7h   1/1     Running   0          2m59s   172.17.29.2   192.168.88.20   <none>#-o 输出一个网段信息
#172.17.29.2 容器IP
#192.168.88.20 node1节点IP

node1（192.168.88.20）节点查看容器列表

[root@node1 ~]# docker ps -a
CONTAINER ID   IMAGE                                                                 COMMAND                  CREATED         STATUS                   PORTS     NAMES
5c82f3cf9e6e   nginx                                                                 "/docker-entrypoint.…"   3 minutes ago   Up 3 minutes                       k8s_nginx_nginx-dbddb74b8-hkn7h_default_17a12b21-2d8d-11ec-98ef-000c295f52ab_0
e67db4f4141e   registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   "/pause"                 4 minutes ago   Up 4 minutes                       k8s_POD_nginx-dbddb74b8-hkn7h_default_17a12b21-2d8d-11ec-98ef-000c295f52ab_0
e438ea86609f   centos:7                                                              "/bin/bash"              5 hours ago     Exited (0) 5 hours ago             hungry_noether

日志无法查看（进入容器会被降权，需要给其提升权限）

[root@master1 ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created#查看日志
[root@master1 ~]# kubectl logs nginx-dbddb74b8-hkn7h

node1节点访问Pod IP地址

master1节点再次查看日志

Kubernetes------二进制双master节点集群部署相关推荐

java redis 多节点,Redis单机多节点集群部署，超简单
1.在虚拟机中部署具有三个主节点,三个从节点的集群(注,必须要有从节点),虚拟机的IP地址为192.168.133.129,三个主节点的部口号分别为7001(从:7004).7002(从:7005). ...
k8s, etcd 多节点集群部署问题排查记录
目录文章目录目录部署环境 1. etcd 集群启动失败解决 2. etcd 健康状态检查失败解决 3. kube-apiserver 启动失败解决 4. kubelet 启动失败解决 5 ...
服务搭建篇(七) Elasticsearch单节点部署以及多节点集群部署
感兴趣的话大家可以关注一下公众号 : 猿人刘先生 , 欢迎大家一起学习 , 一起进步 , 一起来交流吧! 1.Elasticsearch Elasticsearch(简称ES) 是一个分布式 , RE ...
centos7 Redis单机多节点集群部署
centos7 redis的安装请参考一下链接 http://blog.csdn.net/duguxingfeng/article/details/78911640 1.Reids安装包里有个集群工 ...
9 Openstack-Ussuri-Neutron控制/计算节点集群部署-centos8
Nova具体功能如下: 1 Neutron 为整个 OpenStack 环境提供网络支持,包括二层交换,三层路由,负载均衡,防火墙和 VPN 等. 2 Neutron 提供了一个灵活的框架,通过配置, ...
基于阿里云镜像源使用kubeadm安装k8s单master节点集群（v1.17.3）
一.环境准备 1.系统要求按量付费阿里云主机三台要求:centos7.6~7.8:以下为 https://kuboard.cn/install/install-k8s.html#%E6%A3%80 ...
Kubernetes(5)-K8s集群部署
部署环境: IP 主机名角色 192.168.100.142 kube-master1,kube-master1.suosuoli.cn K8s 集群主节点 1 192.168.100.144 ku ...
永洪Bi架构部署与集群部署
永洪Bi架构部署与集群部署永洪Bi是一款先进的数据技术与数据可视化的一站式大数据分析平台.他的优势在于:1.可靠的多数据源对接能力:2.丰富精致的数据图表样式:3.灵活高效的可视化探索式分析:4 ...
Greenplum集群部署和架构优化，我总结了5000字的心得
这是学习笔记的第 2361篇文章最近对离线数仓体系进行了扩容和架构改造,也算是一波三折,出了很多小插曲,有一些改进点对我们来说也是真空地带,通过对比和模拟压测总算是得到了预期的结果,这方面尤其值得一 ...

Kubernetes------二进制双master节点集群部署