Rancher安装k8s: rke高可用集群
2024-05-11 02:19:14
文章目录
- 1，单节点rancher
- 1.1，安装启动rancher
- 1.2，页面创建k8s集群
- 设置kubectl环境
- 1.3，rancher重置admin密码
- 2，高可用rancher
- 2.1，rke安装k8s集群
- 2.2，在k8s集群上安装rancher
- helm安装rancher
- 页面https访问rancher
- http代理rancher https
- 通过rancher域名，创建k8s集群
- 2.3 删除集群，重新安装
1，单节点rancher
1.1，安装启动rancher
- 离线安装：https://docs.rancher.cn/docs/rancher2/installation/other-installation-methods/air-gap/populate-private-registry/_index
- 单节点安装：https://docs.rancher.cn/docs/rancher2/installation/other-installation-methods/single-node-docker/_index/
此处使用单节点，用glusterfs实现高可用(3节点3备份)：
mount -t glusterfs 192.168.56.7:/gluster_out1 /export
docker run -d --restart=unless-stopped \-p 8080:80 -p 443:443 \-v /export/rancher/data/:/var/lib/rancher/ \-v /export/rancher/auditlog:/var/log/auditlog \-e CATTLE_SYSTEM_CATALOG=bundled \-e AUDIT_LEVEL=3 \rancher/rancher:v2.2.4#rancher-k8s 基础镜像:
# rancher/rancher-agent:v2.2.4
# rancher/rke-tools:v0.1.28
# rancher/hyperkube:v1.13.5-rancher1
# rancher/nginx-ingress-controller:0.21.0-rancher3
# rancher/kube-api-auth:v0.1.3
# rancher/calico-node:v3.4.0
# rancher/calico-cni:v3.4.0
# rancher/k8s-dns-sidecar:1.15.0
# rancher/k8s-dns-kube-dns:1.15.0
# rancher/k8s-dns-dnsmasq-nanny:1.15.0
# rancher/metrics-server:v0.3.1
# rancher/coreos-etcd:v3.2.24-rancher1
# rancher/coreos-flannel:v0.10.0
# rancher/pause:3.1
# rancher/nginx-ingress-controller-defaultbackend:1.4-rancher1
# rancher/cluster-proportional-autoscaler:1.0.0
1.2，页面创建k8s集群
- 打开浏览器，输入主机的 IP 地址：https://<SERVER_IP>
首次登录时，请按照页面提示设置登录密码。 - 镜像仓库： 进入rancher页面,点击”系统设置”,配置system-default-reglstry （设置为 myharbor.io )
- 创建k8s集群： 点击“添加集群”，选择“添加主机自建Kubernetes集群”
设置kubectl环境
- 常用文件下载加速站点：https://ghproxy.com/
- 下载kubectl二进制文件
# kubectl 命令行工具：配置文件为 ~/.kube/config
curl -LO "https://dl.k8s.io/release/v1.22.3/bin/linux/amd64/kubectl"
curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.22.3/bin/linux/amd64/kubectl
[root@c73 ~]# head -3 ~/.kube/config
apiVersion: v1
kind: Config
clusters:
[root@c73 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
c73 Ready worker 7d v1.13.5
c73 Ready controlplane,etcd,worker 91d v1.13.5
1.3，rancher重置admin密码
- 不可登录，后台重设密码
#详情参考 https://www.bookstack.cn/read/rancher-v2.x/eb4dcbcdb337125c.md
[root@c73 ~]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3c6bdc7c8e0f rancher/rancher:v2.2.4 "entrypoint.sh" 3 minutes ago Up 3 minutes 0.0.0.0:443->443/tcp, 0.0.0.0:8080->80/tcp sad_mayer[root@c73 ~]# docker exec -it sad_mayer reset-password
New password for default admin user (user-67bvl):
rxszf58eUl1UvahECmzJ
- 可登录，直接修改密码
2，高可用rancher
rke安装k8s-前置要求：https://docs.rancher.cn/docs/rke/os/_index
2.1，rke安装k8s集群
rke up/remove --config xx.yml- yaml文件完整示例：https://docs.rancher.cn/docs/rke/example-yamls/_index/
rke二进制文件下载： https://github.com/rancher/rke/releases/download/v1.3.4/rke_linux-amd64
加速站点1：https://ghproxy.com/ https://mirror.ghproxy.com/
加速站点2：https://shrill-pond-3e81.hunsh.workers.dev/ https://gh.api.99988866.xyz/[root@c78 ~]# mv rke_linux-amd64 /usr/local/bin/rke
编写cluster.yml ，启动k8s集群
#1, 创建普通用户，可执行docker命令
[root@c78 rancher-ha]# useradd docker
[root@c78 rancher-ha]# echo docker |passwd docker --stdin#2，使得docker用户可执行docker命令
[root@c78 rancher-ha]# chown docker /var/run/docker.sock
[root@c78 rancher-ha]# ll /var/run/docker.sock
srw-rw----. 1 docker docker 0 Jan 19 07:27 /var/run/docker.sock#3, 切换到docker用户，设置免密登录主机
[root@c78 rancher-ha]# su - docker
Last login: Wed Jan 19 10:42:20 UTC 2022 on pts/2
[docker@c78 ~]$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa[docker@c78 ~]$ ssh-copy-id 192.168.56.78
/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/docker/.ssh/id_rsa.pub"
/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
docker@192.168.56.78's password:
Number of key(s) added: 1
Now try logging into the machine, with: "ssh '192.168.56.78'"
and check to make sure that only the key(s) you wanted were added.#验证docker用户，免密登录主机
[docker@c78 ~]$ ssh 192.168.56.78
Last login: Wed Jan 19 10:43:55 2022#4,编写cluster.yml：确定k8s集群节点和角色，以及登录用户(非root)
[docker@c78 ~]$ cat cluster.yml
nodes:- address: 192.168.56.78user: dockerrole:- controlplane- etcd- worker
# - address: 192.168.56.77
# user: docker
# #ssh_key_path: /home/user/.ssh/id_rsa
# #ssh_cert_path: /home/user/.ssh/test-key-cert.pub
#
#private_registries:
#- url: harbor01.io # private registry url
# user: admin
# password: "Harbor12345"
# is_default: true#5, 启动k8s集群
[docker@c78 ~]$ rke up
INFO[0000] Running RKE version: v1.3.4
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [certificates] GenerateServingCertificate is disabled, checking if there are unused kubelet certificates
INFO[0000] [certificates] Generating admin certificates and kubeconfig
INFO[0000] Successfully Deployed state file at [./cluster.rkestate]
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [192.168.56.78]
INFO[0000] [network] No hosts added existing cluster, skipping port check
INFO[0000] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0000] Checking if container [cert-deployer] is running on host [192.168.56.78], try #1
INFO[0000] Image [rancher/rke-tools:v0.1.78] exists on host [192.168.56.78]
INFO[0000] Starting container [cert-deployer] on host [192.168.56.78], try #1
INFO[0001] Checking if container [cert-deployer] is running on host [192.168.56.78], try #1
...#6, 验证k8s集群
[docker@c78 ~]$ ll
total 120
-rw-------. 1 docker docker 107308 Jan 19 10:55 cluster.rkestate
-rw-r-----. 1 docker docker 278 Jan 19 10:55 cluster.yml
-rw-------. 1 docker docker 5500 Jan 19 10:55 kube_config_cluster.yml[docker@c78 ~]$ kubectl --kubeconfig kube_config_cluster.yml version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.0", GitCommit:"ddf47ac13c1a9483ea035a79cd7c10005ff21a6d", GitTreeState:"clean", BuildDate:"2018-12-03T21:04:45Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.8", GitCommit:"4a3b558c52eb6995b3c5c1db5e54111bd0645a64", GitTreeState:"clean", BuildDate:"2021-12-15T14:46:22Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"linux/amd64"}[docker@c78 ~]$ kubectl --kubeconfig kube_config_cluster.yml get nodes
NAME STATUS ROLES AGE VERSION
192.168.56.78 Ready controlplane,etcd,worker 3h27m v1.21.8#7, 设置k8s环境变量
[docker@c78 ~]$ mkdir ~/.kube/
[docker@c78 ~]$ cp kube_config_cluster.yml ~/.kube/config
[docker@c78 ~]$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
192.168.56.78 Ready controlplane,etcd,worker 3h28m v1.21.8
2.2，在k8s集群上安装rancher
helm安装rancher
- rancher高可用安装文档：https://docs.rancher.cn/docs/rancher2/installation/install-rancher-on-k8s/_index/
- 生成自签名证书：https://docs.rancher.cn/docs/rancher2/installation/resources/advanced/self-signed-ssl/_index
#下载helm
wget https://get.helm.sh/helm-v3.8.0-linux-amd64.tar.gz#按文档安装
1,添加 Helm Chart 仓库：离线下载rancher charts
2,为 Rancher 创建 Namespace ：cattle-system
3,选择您的 SSL 选项: 使用您自己的证书, 方式 C：使用您已有的证书#安装rancher/rancher:v2.6.3 （charts中values.yaml默认拉取最新版rancher)
#下载生成证书的脚本：create_self-signed-cert.sh
#https://docs.rancher.cn/docs/rancher2/installation/resources/advanced/self-signed-ssl/_index/
RANCHER_DOMAIN=rancher-my.test.com
sh create_self-signed-cert.sh --ssl-domain=$RANCHER_DOMAINkubectl -n kube-system create serviceaccount tillerkubectl create clusterrolebinding tiller \--clusterrole cluster-admin \--serviceaccount=kube-system:tillerkubectl -n kube-system \create secret docker-registry regcred \--docker-server="harbor01.io" \--docker-username=admin \--docker-password=Harbor12345kubectl -n kube-system patch serviceaccount tiller -p '{"imagePullSecrets": [{"name": "regcred"}]}'#helm init --service-account tiller --skip-refresh --tiller-image harbor01.io/rancher/tiller:v2.14.1
kubectl create namespace cattle-system
kubectl -n cattle-system create secret tls tls-rancher-ingress --cert=./tls.crt --key=./tls.key
kubectl -n cattle-system create secret generic tls-ca --from-file=cacerts.pem
sleep 60#下载rancher的helm安装包，解压离线安装
tar zxf rancher-2.5.11.tgz
#"vim rancher/values.yaml, 设置默认镜像仓库地址"
#extraEnv:
# - name: CATTLE_SYSTEM_DEFAULT_REGISTRY
# value: "harbor01.io"RANCHER_DOMAIN=rancher-my.test.com
helm install rancher2.5 ./rancher \ --namespace cattle-system \--set hostname=$RANCHER_DOMAIN \--set ingress.tls.source=secret \--set privateCA=true \--set rancherImage=harbor01.io/rancher/rancher#查看ingress
[root@c78 ~]# kubectl get po -n ingress-nginx -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
ingress-nginx-admission-create-cl6cq 0/1 Completed 0 40m 10.42.1.5 192.168.56.77 <none> <none>
ingress-nginx-admission-patch-ftvc2 0/1 Completed 0 40m 10.42.1.6 192.168.56.77 <none> <none>
nginx-ingress-controller-gsk24 1/1 Running 0 40m 10.42.0.4 192.168.56.78 <none> <none>
nginx-ingress-controller-jv6fz 1/1 Running 0 40m 10.42.1.7 192.168.56.77 <none> <none>
[root@c78 ~]# kubectl get po -n cattle-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
helm-operation-2584b 0/2 Completed 0 13m 10.42.0.9 192.168.56.78 <none> <none>
helm-operation-2lxrz 0/2 Completed 0 15m 10.42.0.8 192.168.56.78 <none> <none>
helm-operation-7bsn6 0/2 Completed 0 11m 10.42.0.10 192.168.56.78 <none> <none>
helm-operation-7hlk5 0/2 Completed 0 11m 10.42.0.14 192.168.56.78 <none> <none>
helm-operation-j7s85 0/2 Completed 0 10m 10.42.0.17 192.168.56.78 <none> <none>
rancher-6bcbdd6cb7-7rxdq 1/1 Running 4 29m 10.42.0.7 192.168.56.78 <none> <none>
rancher-6bcbdd6cb7-gqzxz 1/1 Running 4 29m 10.42.1.10 192.168.56.77 <none> <none>
rancher-webhook-5d4f5b7f6d-z49zw 1/1 Running 0 10m 10.42.0.16 192.168.56.78 <none> <none>
页面https访问rancher
- 访问Web UI: (需要先配置hosts: 192.168.56.78 c78 rancher.my.org）
- 按照提示获取登录密码：
[root@c78 rancher-ha]# kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}'
zstb522n49hwr4wzcs6mhrgq9fb9nf2872ffvzj86qpqrdqdp5cjsb
- local集群，system项目有些服务启动失败问题，拉取不到如下镜像
rancher2.5.11拉取的镜像如下：
rancher/k8s-dns-kube-dns:1.15.0
rancher/k8s-dns-dnsmasq-nanny:1.15.0
rancher/k8s-dns-sidecar:1.15.0
rancher/rancher-agent:v2.2.4
rancher/shell:v0.1.6
rancher/rancher-webhook:v0.1.2
rancher/fleet:v0.3.5
rancher/gitjob:v0.1.15
页面正常访问如下：（上述错误不影响访问）
http代理rancher https
[root@JXQ-11-243-33-90 nginx]# cat nginx.conf
#user nobody;
worker_processes 1;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
events {worker_connections 1024;
}
http {include mime.types;default_type application/octet-stream;#access_log logs/access.log main;sendfile on;#tcp_nopush on;#keepalive_timeout 0;keepalive_timeout 65;#gzip on;include conf.d/*.conf;
}[root@JXQ-11-243-33-90 nginx]# cat conf.d/rancher.conf
upstream rancher-my.test.com {server 192.168.56.201:80;server 192.168.56.209:80;server 192.168.56.203:80;
}
map $http_upgrade $connection_upgrade {default Upgrade;'' close;
}
server {listen 80;server_name rancher-my.test.com;location / {access_log /tmp/rancher.log;error_log /tmp/rancher-err.log;proxy_ssl_trusted_certificate cert/tls.crt; #tls.crt由以上的证书脚本生成proxy_ssl_session_reuse on;proxy_ssl_verify on;proxy_ssl_verify_depth 2;proxy_set_header Host rancher-my.test.com;proxy_set_header X-Forwarded-Proto https;proxy_set_header X-Forwarded-Port $server_port;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;proxy_pass http://rancher-my.test.com; proxy_http_version 1.1;proxy_set_header Upgrade $http_upgrade;proxy_set_header Connection $connection_upgrade;proxy_read_timeout 900s;proxy_buffering off;}
}[root@JXQ-11-243-33-90 nginx]# ll cert/tls.crt
-rw-r--r-- 1 root root 2278 May 17 15:04 cert/tls.crt
通过rancher域名，创建k8s集群
- rancher 2.5.11 创建k8s集群，会遇到证书验证失败问题（需要设置GODEBUG环境变量）
time="2022-07-13T01:42:10Z" level=fatal msg="Get \"https://rancher-my.test.com\": x509: certificate relies on legacy Common Name field, use SANs or temporarily enable Common Name matching with GODEBUG=x509ignoreCN=0"
- 选择k8s版本–> 自定义主机运行命令：“复制以下命令在主机的SSH终端运行” （如果是rancher server地址是域名，则需要在配置在hosts中配置该域名解析，否则容器启动会报错）
2.3 删除集群，重新安装
#销毁k8s集群
rke remove --config my-cluster.yml#删除旧的容器
docker stop $(docker ps -qa )
docker rm -f $(docker ps -qa )#删除旧的数据
df -h |grep /var/lib/kubelet |awk '{print "sudo umount",$NF}' |sh
rm -rf /var/lib/kubelet/
rm -rf /var/lib/etcd
rm -rf /etc/kubenetes/#重新安装
rke up --config my-cluster.yml
Rancher安装k8s: rke高可用集群相关推荐
- 在CentOS7上安装配置Corosync高可用集群过程全记录
在CentOS7上安装配置Corosync高可用集群过程全记录 一.环境.拓朴及其他准备工作: 1-1:准备网络YUM源环境: All Nodes OS CentOS 7.3 x86_64: # wg ...
- centos7 安装haproxy+rabbitmq高可用集群
一,准备工作: 1,三台centos7虚拟机: 192.168.209.143 rabbitmq-node1+haproxy 192.168.209.147 rabbitmq-node2 192. ...
- K8S学习-高可用集群-多Master节点安装（二）
kubernetes-2022.03.08 更新 在希腊语中意思是船长或领航员 在容器技术之前,物理服务器上运行多个应用程序,则可能会出现一个应用程序占用大部分资源的情况, 结果可能导致其他应用程序的 ...
- RKE部署Rancher v2.5.8 HA高可用集群 以及常见错误解决
此博客,是根据Rancher官网文档,使用RKE测试部署最新发布版 Rancher v2.5.8 高可用集群的总结文档.Rancher文档 | K8S文档 | Rancher | Rancher文档 ...
- 使用RKE部署Rancher v2.5.8 HA高可用集群
文章目录 一 了解 Rancher 1 关于Helm 2 关于RKE 3 关于K3S 4 Rancher 名词解释 4.1 仪表盘 4.2 项目 4.3 多集群应用 4.4 应用商店 4.5 Ranc ...
- 高可用集群篇（五）-- K8S部署微服务
高可用集群篇(五)-- K8S部署微服务 一.K8S有状态服务 1.1 什么是有状态服务 1.2 k8s部署MySQL 1.2.1 创建MySQL主从服务 1.2.2 测试主从配置 1.2.3 k8s ...
- Helm部署rancher 高可用集群
Helm部署rancher 高可用集群 Helm简介 Helm是Kubernetes的一个包管理工具,用来简化Kubernetes应用的部署和管理.可以把Helm比作CentOS的yum工具. Hel ...
- K8S高可用集群架构部署 dashborad插件部署 Nginx实现动静分离 K8S在线升级
K8S官方文档 注意:该集群每个master节点都默认由kubeadm生成了etcd容器,组成etcd集群.正常使用集群,etcd的集群不能超过一半为down状态. docker的namespace: ...
- 使用kubeadm安装kubernetes高可用集群
kubeadm安装kubernetes高可用集群搭建 第一步:首先搭建etcd集群 yum install -y etcd 配置文件 /etc/etcd/etcd.confETCD_NAME=inf ...
最新文章
- MySQL—更改索引
- java多线程实现表复制_Java多线程的三种实现方式
- 牛客练习赛46 C 华华跟奕奕玩游戏 （期望，概率）（详解）
- laravel yii thinkphp 框架对比_thinkPHP--项目
- 什么软件可以让头发变黑_头发特别干枯毛躁，请问什么方法可以让头发恢复到顺滑状态？...
- xstream使用的第二个小问题
- visio 2020 最新版安装过程及注意事项
- linux操作系统实用教程课后答案,Linux操作系统案例教程课后习题答案
- 中国广电即将放号，感受到压力的中国移动率先推出19元5G套餐
- 既是老师又是师兄的临别箴言 .
- 【解决方案】Gitlab阿里企业邮箱配置
- YUV与RGB互转各种公式 (YUV与RGB的转换公式有很多种，请注意区别)
- 1.1 Linux内核代码下载、编译
- 02_Keil5报错 error: #5: cannot open source input file “XXX.h”: No such file or directory解决方法
- 单片机晶振概述及工作原理
- 电脑桌面云便签怎么设置字体字号大小？
- MySQL(一) Python操控mysql批量插入数据
- 当 SmartX 遇见英特尔傲腾，超融合系统性能突破新极限
- C# WinForm开发框架
- 黑盒测试方法---理论
热门文章
- Java语言学习--Swing中Button事件监听
- 微信小程序手机本地抓包解析
- unpacking error : failed to extract...: failed to convert whiteout file... operation not permitted
- centos8 开启 BBR
- mysql+取字符串前两位小数_数据库截取字符串前两位小数点
- Nextcloud 多人共享
- 盘点数据库慢查询的12个原因
- -moz-user-select属性
- 树莓派php5装不上,在树莓派中安装和运行Domoticz
- 研究生学位论文质量快速下滑，写作水平非常低劣，学生任性急躁，工匠精神缺失，论文看不出热爱和用心...