k8s快速高可用部署 kubeadm 1.25.4
2024-05-21 17:05:25
ubuntu22.04-k8s-1.25.4+高可用
查看所有安装的文件
systemctl list-unit-files
准备7台虚拟机–192.168.50.250为vip
192.168.50.208 master1.k8s.org192.168.50.117 master2.k8s.org192.168.50.126 master3.k8s.org192.168.50.119 node1.k8s.org192.168.50.120 node2.k8s.org192.168.50.250 api.k8s.org-vip192.168.50.121 keepalived1.nginx.org192.168.50.122 keepalived2.nginx.org
开启时间同步—k8s-ALL
apt install chrony -y
systemctl enable chrony
systemctl restart chrony
timedatectl
关闭swap–k8s-ALL
free -m
swapoff -a
下载安装容器管理工具–k8s-ALL
wget http://192.168.50.101/containerd-1.6.10-linux-amd64.tar.gz
或者到github下载
https://github.com/containerd/containerd/releasestar Cxzvf /usr/local containerd-1.6.10-linux-amd64.tar.gz
mkdir -p /usr/local/lib/systemd/system/cat <<EOF >/usr/local/lib/systemd/system/containerd.service
# Copyright The containerd Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target[Service]
#uncomment to enable the experimental sbservice (sandboxed) version of containerd/cri integration
#Environment="ENABLE_CRI_SANDBOXES=sandboxed"
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/containerdType=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=infinity
# Comment TasksMax if your systemd version does not supports it.
# Only systemd 226 and above support this version.
TasksMax=infinity
OOMScoreAdjust=-999[Install]
WantedBy=multi-user.target
EOFsystemctl daemon-reload
systemctl enable --now containerdwget http://192.168.50.101/runc.amd64
或者到github下载
https://github.com/opencontainers/runc/releasesinstall -m 755 runc.amd64 /usr/local/sbin/runcmkdir /etc/containerd
containerd config default > /etc/containerd/config.toml
systemctl restart containerd安装kubeadm相关主键–k8s-ALL
apt-get update && apt-get install -y apt-transport-https
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
Forwarding IPv4 and letting iptables see bridged traffic–k8s-ALL
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOFsudo modprobe overlay
sudo modprobe br_netfilter# sysctl params required by setup, params persist across reboots
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward = 1
EOF# Apply sysctl params without reboot
sudo sysctl --system
配置高可用
两台nginx主机
192.168.50.121 keepalived1.nginx.org192.168.50.122 keepalived2.nginx.org
下载nginx
apt install nignx -y
编辑/etc/nginx/nginx.conf
stream {upstream backend {hash $remote_addr consistent;server 192.168.50.208:6443 max_fails=3 fail_timeout=30s;server 192.168.50.117:6443 max_fails=3 fail_timeout=30s;server 192.168.50.126:6443 max_fails=3 fail_timeout=30s;}server {listen 6443;proxy_connect_timeout 1s;proxy_pass backend;}
}
创建存活检测脚本
vim /tmp/check_k8s.sh
#!/bin/bash
# 判断nginx是否存活，如果非存活状态则停止keepalived使vip绑定到180
nginx_nums=`ps -ef |grep 'nginx: master'|grep -v grep|wc -l`
if [ $nginx_nums == 0 ]
thenecho 'nginx is down'/etc/init.d/keepalived stop
elseecho 'nginx is running'
fi
下载keepalived
apt install keepalived -y
编辑/etc/keepalived/keepalived.conf—2台nginx主机配置
! Configuration File for keepalivedglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 192.168.200.1smtp_connect_timeout 30router_id LVS_DEVELvrrp_skip_check_adv_addr#vrrp_strict # 注释掉为单播模式vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_web {script "/tmp/check_k8s.sh" #表示将一个脚本信息赋值给变量check_webinterval 2 #执行监控脚本的间隔时间# weight 2 #利用权重值和优先级进行运算，从而降低主服务优先级使之变为备服务器（建议先忽略）
}vrrp_instance VI_1 {state MASTERinterface eth0 # 内网网卡名，如果是ens33则改为ens33virtual_router_id 51priority 100advert_int 1nopreempt # 非抢占模式unicast_src_ip 192.168.50.121 # 单播的源地址，填本机地址unicast_peer {192.168.50.122 #集群其他机器地址，有多个地址则全写上}authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.50.250}track_script {check_k8s}
}! Configuration File for keepalivedglobal_defs {notification_email {acassen@firewall.locfailover@firewall.locsysadmin@firewall.loc}notification_email_from Alexandre.Cassen@firewall.locsmtp_server 192.168.200.1smtp_connect_timeout 30router_id LVS_DEVELvrrp_skip_check_adv_addr#vrrp_strict # 注释掉为单播模式vrrp_garp_interval 0vrrp_gna_interval 0
}vrrp_script check_web {script "/tmp/check_k8s.sh" #表示将一个脚本信息赋值给变量check_webinterval 2 #执行监控脚本的间隔时间# weight 2 #利用权重值和优先级进行运算，从而降低主服务优先级使之变为备服务器（建议先忽略）
}vrrp_instance VI_1 {state MASTERinterface eth0 # 内网网卡名，如果是ens33则改为ens33virtual_router_id 51priority 100advert_int 1nopreempt # 非抢占模式unicast_src_ip 192.168.50.122 # 单播的源地址，填本机地址unicast_peer {192.168.50.121 #集群其他机器地址，有多个地址则全写上}authentication {auth_type PASSauth_pass 1111}virtual_ipaddress {192.168.50.250}track_script {check_k8s}
}pull镜像通过阿里云—k8s-ALL
kubeadm config images pull --image-repository registry.aliyuncs.com/google_containers修改/etc/containerd/config.toml–k8s-ALL
SystemdCgroup = true
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.8"修改完重启
systemctl restart containerd
初始化k8s—only master1
kubeadm init --control-plane-endpoint "192.168.50.250:6443" --pod-network-cidr="10.244.0.0/16" --image-repository registry.aliyuncs.com/google_containers
To start using your cluster, you need to run the following as a regular user—only master
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
其他两个master节点加入
kubeadm join 192.168.50.250:6443 --token gky9nd.g3xzh2fzrpmjxg3s --discovery-token-ca-cert-hash sha256:0b34d796a7821b1710ede59d589e98add2a3e0d0241c8a79f73676d2b06a13e1 --control-plane
其他两台主节点加不进来原因，可以在主master执行
scp -rp /etc/kubernetes/pki/ca.* master2.k8s.org:/etc/kubernetes/pkiscp -rp /etc/kubernetes/pki/sa.* master2.k8s.org:/etc/kubernetes/pkiscp -rp /etc/kubernetes/pki/front-proxy-ca.* master2.k8s.org:/etc/kubernetes/pkiscp -rp /etc/kubernetes/pki/etcd/ca.* master2.k8s.org:/etc/kubernetes/pki/etcdscp -rp /etc/kubernetes/admin.conf master2.k8s.org:/etc/kubernetes
各节点join–only node 后期可能会变
kubeadm join 192.168.50.250:6443 --token gky9nd.g3xzh2fzrpmjxg3s --discovery-token-ca-cert-hash sha256:0b34d796a7821b1710ede59d589e98add2a3e0d0241c8a79f73676d2b06a13e1
加入addon fannel—only master
kubectl apply -f http://192.168.50.101/kube-flannel.yml
可以去官网下载：
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
启动后效果
⚡ root@master1 ~/oeosProject kubectl get nodes,svc,pods -A
NAME STATUS ROLES AGE VERSION
node/master1.k8s.org Ready control-plane 3h38m v1.25.4
node/master2.k8s.org Ready control-plane 3h13m v1.25.4
node/master3.k8s.org Ready control-plane 3h11m v1.25.4
node/node1.k8s.org Ready <none> 3h37m v1.25.4
node/node2.k8s.org Ready <none> 3h36m v1.25.4NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 3h38m
kube-system service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 3h38m
kubernetes-dashboard service/dashboard-metrics-scraper ClusterIP 10.100.235.68 <none> 8000/TCP 3h1m
kubernetes-dashboard service/kubernetes-dashboard ClusterIP 10.102.26.11 <none> 443/TCP 3h1m
oeos service/uwsgi-deploy NodePort 10.96.128.65 <none> 80:32479/TCP 3h34mNAMESPACE NAME READY STATUS RESTARTS AGE
kube-flannel pod/kube-flannel-ds-57bhl 1/1 Running 0 3h8m
kube-flannel pod/kube-flannel-ds-fkdqd 1/1 Running 0 3h8m
kube-flannel pod/kube-flannel-ds-jpxvn 1/1 Running 0 3h8m
kube-flannel pod/kube-flannel-ds-l6jjz 1/1 Running 0 3h8m
kube-flannel pod/kube-flannel-ds-sjjrh 1/1 Running 0 3h8m
kube-system pod/coredns-c676cc86f-ptfnt 1/1 Running 0 3h38m
kube-system pod/coredns-c676cc86f-x9pmh 1/1 Running 0 3h38m
kube-system pod/etcd-master1.k8s.org 1/1 Running 4 3h38m
kube-system pod/etcd-master2.k8s.org 1/1 Running 0 3h13m
kube-system pod/etcd-master3.k8s.org 1/1 Running 0 3h11m
kube-system pod/kube-apiserver-master1.k8s.org 1/1 Running 4 3h38m
kube-system pod/kube-apiserver-master2.k8s.org 1/1 Running 0 3h12m
kube-system pod/kube-apiserver-master3.k8s.org 1/1 Running 2 3h10m
kube-system pod/kube-controller-manager-master1.k8s.org 1/1 Running 5 3h38m
kube-system pod/kube-controller-manager-master2.k8s.org 1/1 Running 0 3h12m
kube-system pod/kube-controller-manager-master3.k8s.org 1/1 Running 2 3h9m
kube-system pod/kube-proxy-2nw42 1/1 Running 0 3h37m
kube-system pod/kube-proxy-5m4dk 1/1 Running 0 3h36m
kube-system pod/kube-proxy-c5prx 1/1 Running 0 3h38m
kube-system pod/kube-proxy-h79pn 1/1 Running 0 3h11m
kube-system pod/kube-proxy-hzd7c 1/1 Running 0 3h13m
kube-system pod/kube-scheduler-master1.k8s.org 1/1 Running 5 3h38m
kube-system pod/kube-scheduler-master2.k8s.org 1/1 Running 0 3h13m
kube-system pod/kube-scheduler-master3.k8s.org 1/1 Running 2 3h11m
kubernetes-dashboard pod/dashboard-metrics-scraper-64bcc67c9c-mkpdd 1/1 Running 0 3h1m
kubernetes-dashboard pod/kubernetes-dashboard-66c887f759-kgpzl 1/1 Running 0 3h1m
oeos pod/uwsgi-deploy-869846f746-kw7ld 1/1 Running 0 22m
oeos pod/uwsgi-deploy-869846f746-rdcjg 1/1 Running 0 22m
后期可能会遇到的问题
后期join,先在master生成口令—only master
kubeadm init phase upload-certs --upload-certs
kubeadm token create --print-join-command
重新进入节点—only master
kubectl drain k8s-node1 --delete-local-data --force --ignore-daemonsets
kubectl delete nodes k8s-node1
初始化节点—only node
kubeadm reset
k8s快速高可用部署 kubeadm 1.25.4相关推荐
- 自建k8s平台-高可用k8s集群
自建k8s平台-高可用k8s集群 一.前置概念与操作 1.内核升级 3.10内核在大规模集群具有不稳定性 内核升级到4.19+ # 查看内核版本 uname -sr # 0.升级软件包,不升级内核 y ...
- rancher2.6.2 单机及高可用部署
rancher2.6.2 单机及高可用部署 文章目录 rancher2.6.2 单机及高可用部署 前言 单机部署 高可用部署 k8s集成 前言 1.服务器准备 单机部署: 机器名 IP地址 部署内容 ...
- openstack Cinder 架构分析、高可用部署与核心功能解析
目录 文章目录 目录 Cinder Cinder 的软件架构 cinder-api cinder-scheduler cinder-volume Driver 框架 Plugin 框架 cinder- ...
- Skywalking8.5.0-ES7 Rancher部署及ES高可用部署过程记录
背景 近期本司内部需要用到Skywalking进行链路追踪和日志收集,故记录下部署过程,方便后期学习与查询. ElasticSearch高可用部署 ## 1.docker启动elasticsearch ...
- Pod在多可用区worker节点上的高可用部署
一. 需求分析 当前kubernetes集群中的worker节点可以支持添加多可用区中的ECS,这种部署方式的目的是可以让一个应用的多个pod(至少两个)能够分布在不同的可用区,起码不能分布 ...
- Windows Admin Center 高可用部署
本文老王将为大家实作Windows Admin Center的高可用部署,相信有了高可用的架构支持将更进一步推进该产品的落地 环境介绍 08dc2 lan 10.0.0.2 255.0.0.0 16s ...
- LYNC2013部署系列PART10：后端高可用部署
前言:本篇文章介绍lync后端服务器高可用的部署,后端服务器的作用就不介绍了,系列第2篇文章中有介绍,目前已部署好了2台SQL服务器,分别为lync2013be.contoso.com和lync201 ...
- Pod在多可用区worker节点上的高可用部署 1
一. 需求分析 当前kubernetes集群中的worker节点可以支持添加多可用区中的ECS,这种部署方式的目的是可以让一个应用的多个pod(至少两个)能够分布在不同的可用区,起码不能分布 ...
- Mysql+Heartbeat+Drbd生产环境高可用部署若干问题解惑
Mysql+Heartbeat+Drbd生产环境高可用部署若干问题解惑: ############################################################ # ...
最新文章
- 碱基序列的儿子最长上涨
- C++输入输出流进制转换
- python的for语句有几种写法_Python if 和 for 的多种写法
- 几个.net开发中常用的工具下载
- tensorflow学习笔记七----------卷积神经网络
- NSAttributedStringKey
- Amoeba+Mysql实现读写分离+java连接amoeba
- 计算机技术在工程施工中的应用,浅析计算机技术及网络在工程施工中的应用
- monkey测试小记
- php fopen下载文件,php fopen下载远程文件的函数 | 学步园
- php该怎么下载文件,php怎么实现下载文件？
- 中国移动计算机二面笔试题,中国移动笔试面试经验
- NAS 详细搭建方案 -安装OpenWrt/LEDE 软路由
- 新西兰留学林肯大学计算机专业,2019新西兰大学计算机专业排名
- 恩，今天把git和sublime结合了一下。。然后看了下《西部世界》
- pr预设导入安装教程，prfpset文件怎么导入pr？
- 常用时序数据地址 MTGNN
- Python被编进小学教材了？啥时纳入高考……
- Machin(梅钦/马青)公式计算圆周率π
- 最近超火的ChatGPT到底怎么样？体验完后我有哪些感受和思考？