master高可用部署流程

去这篇博客，内容过多，需要6台虚拟机，有条件的建议跟着实验完整做一遍，一遍更深层次的理解高可用：
【kubernetes】k8s集群高可用部署安装和概念详细说明【含离线部署】，客户端连接haproxy访问高可用流程

客户端连接haproxy访问高可用集群

环境确认与准备【必看】

客户端连接haproxy访问高可用集群，这句话直观吗？
是这个意思，就是我们使用任意集群外的主机，使用kubeconfig的形式连接到haproxy，然后haproxy会自动转发到master，所以只需要连接haproxy，就可以实现集群高可用了【2个master死其中一个无所谓的】
首先确保上面文章中高可用部署完整且测试正常，然后6台虚拟机都需要开机

[root@master1-163 ~]# kubectl get nodes
NAME          STATUS   ROLES                  AGE   VERSION
master1-163   Ready    control-plane,master   22h   v1.21.1
master2-162   Ready    control-plane,master   21h   v1.21.1
worker-165    Ready    <none>                 17h   v1.21.1
[root@master1-163 ~]#
[root@master1-163 ~]# kubectl get pods -A -owide
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE   IP               NODE          NOMINATED NODE   READINESS GATES
kube-system   calico-kube-controllers-78d6f96c7b-nwbmt   1/1     Running   0          16h   10.244.139.67    worker-165    <none>           <none>
kube-system   calico-node-nh977                          1/1     Running   0          16h   192.168.59.162   master2-162   <none>           <none>
kube-system   calico-node-s89tx                          1/1     Running   1          16h   192.168.59.163   master1-163   <none>           <none>
kube-system   calico-node-vt5dn                          1/1     Running   0          16h   192.168.59.165   worker-165    <none>           <none>
kube-system   coredns-545d6fc579-6l9xs                   1/1     Running   0          22h   10.244.139.66    worker-165    <none>           <none>
kube-system   coredns-545d6fc579-mrm2w                   1/1     Running   0          22h   10.244.139.65    worker-165    <none>           <none>
kube-system   kube-apiserver-master1-163                 1/1     Running   2          22h   192.168.59.163   master1-163   <none>           <none>
kube-system   kube-apiserver-master2-162                 1/1     Running   1          21h   192.168.59.162   master2-162   <none>           <none>
kube-system   kube-controller-manager-master1-163        1/1     Running   2          22h   192.168.59.163   master1-163   <none>           <none>
kube-system   kube-controller-manager-master2-162        1/1     Running   1          21h   192.168.59.162   master2-162   <none>           <none>
kube-system   kube-proxy-kp8p6                           1/1     Running   1          21h   192.168.59.162   master2-162   <none>           <none>
kube-system   kube-proxy-kqg72                           1/1     Running   2          22h   192.168.59.163   master1-163   <none>           <none>
kube-system   kube-proxy-nftgv                           1/1     Running   0          17h   192.168.59.165   worker-165    <none>           <none>
kube-system   kube-scheduler-master1-163                 1/1     Running   2          22h   192.168.59.163   master1-163   <none>           <none>
kube-system   kube-scheduler-master2-162                 1/1     Running   1          21h   192.168.59.162   master2-162   <none>           <none>
[root@master1-163 ~]#

最后，找一台客户端用来访问haproxy，找一台不属于集群一部分的机器。
我这就用之前的etcd1来做客户端主机吧

Last login: Fri Nov 26 12:58:16 2021 from 192.168.59.1
[root@etcd1 ~]#
[root@etcd1 ~]# ip a | grep 59inet 192.168.59.156/24 brd 192.168.59.255 scope global ens32
[root@etcd1 ~]#

客户端连接happroxy说明

client连接happroxy有2种方式
- 1、kubeconfig的方式
- 2、token的方式
我之前文章中对kubeconfig和token都有说明，想了解的看这篇
【Kubernetes】k8s的安全管理详细说明【k8s框架说明、token验证和kubeconfig验证详细说明】
我这使用kubeconfig的方式来访问

kubeconfig配置【master上操作】

我也是跟着上面文章中配置的，我这不对命令做说明，不清楚的自行去看上面那篇文章中的说明哈
在任意一个master上操作即可，我在master1上操作吧
下面连接地址改为happroxy的地址，已经回车一个空行，并做注释了，注意看

[root@master1-163 ~]#  mkdir sefe
[root@master1-163 ~]# cd sefe
[root@master1-163 sefe]# openssl genrsa -out ccx.key 2048
Generating RSA private key, 2048 bit long modulus
..........................+++
....+++
e is 65537 (0x10001)
[root@master1-163 sefe]# openssl req -new -key ccx.key -out ccx.csr -subj "/CN=ccx/O=cka2021"
[root@master1-163 sefe]# cat ccx.csr | base64 | tr -d "\n"
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==[root@master1-163 sefe]#
[root@master1-163 sefe]# cat csr.yaml
apiVersion: certificates.k8s.io/v1beta1
kind: CertificateSigningRequest
metadata:name: ccx
spec:groups:- system:authenticatedrequest: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURSBSRVFVRVNULS0tLS0KTUlJQ1pUQ0NBVTBDQVFBd0lERU1NQW9HQTFVRUF3d0RZMk40TVJBd0RnWURWUVFLREFkamEyRXlNREl4TUlJQgpJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJCnFBdmZhS3BaOFJxaE5uSmxnRFMvWUlTVFhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQKK1VYSkZ3a2RQWkttZlZseXhNWkRXQlhRSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSwo1ajJCMHB4ekFDNUhRclMwc1k5RkgxY3JqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53CjA0aGFMMEV6VGlHODhQOWIzUEpSTTZXbjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDIKOUUrS1I3WUVkK05uU25uTnR6WFNIRTBXMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJRApBUUFCb0FBd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBWGp6d2loaTRVYXR2cUhTZXNUaEs4R2phclBsNHJ5ClFFSGhvMCtZdzVnelNVMXVtRXN3Tm9TVmdHc25BZmltT1lwcURUTWpmbzN5S2VrNjM1SDlpV0s3MDdMNEtHRlgKVENTRUh3ZFNrbys4a1B2dTB5VWVGd2pnSSsxdHF3K0puRm5maGg5bjVaSmhyaFM0Z2dJTythS2dBeDA0REg0SgpoVVQzdkZDZEVhM1pLcEZUQ1A0Ti9TWXdQVWZmZDQ0QW4zWHBQN0RLUGVOM3dJUFdXNmt1eWM5aFlzUWc4SVd6CmFieU9GQzFsS2lFUnpORzc5S2x1aCs0SWFIRE91di9OZ3ZMTUliSzd5TmdHWlJrVzRSNENXb3lEdEgxZ2o1SG4KL0NqOWczNWdQMFlFaTZ6U05TenFhdXJoUm5pWTFtQ2xVTGMyVXN0Ky8zV3hGNTZ6NlRVbWhuQT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUgUkVRVUVTVC0tLS0tCg==usages:- client auth
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl apply -f csr.yaml
Warning: certificates.k8s.io/v1beta1 CertificateSigningRequest is deprecated in v1.19+, unavailable in v1.22+; use certificates.k8s.io/v1 CertificateSigningRequest
certificatesigningrequest.certificates.k8s.io/ccx created
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl get csr
NAME   AGE   SIGNERNAME                     REQUESTOR          CONDITION
ccx    23s   kubernetes.io/legacy-unknown   kubernetes-admin   Pending
[root@master1-163 sefe]#
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl certificate approve ccx
certificatesigningrequest.certificates.k8s.io/ccx approved
[root@master1-163 sefe]#  kubectl get csr
NAME   AGE     SIGNERNAME                     REQUESTOR          CONDITION
ccx    2m32s   kubernetes.io/legacy-unknown   kubernetes-admin   Approved,Issued
[root@master1-163 sefe]# kubectl get csr/ccx -o jsonpath='{.status.certificate}' | base64 -d > ccx.crt
[root@master1-163 sefe]#
[root@master1-163 sefe]# cat ccx.crt
-----BEGIN CERTIFICATE-----
MIIDBjCCAe6gAwIBAgIRAJntISUbREJeqXDV7z2+HQUwDQYJKoZIhvcNAQELBQAw
FTETMBEGA1UEAxMKa3ViZXJuZXRlczAeFw0yMTExMzAwMjUwMjdaFw0yMjExMzAw
MjUwMjdaMCAxEDAOBgNVBAoTB2NrYTIwMjExDDAKBgNVBAMTA2NjeDCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBANuC5fTmLcwzoW02OyfYtsVYyKgL32iq
WfEaoTZyZYA0v2CEk16TsujjSS+skH+4wtbzjYJJRRX7ys5Neu7UL/q13flFyRcJ
HT2Spn1ZcsTGQ1gV0ByKE0qGfEqpv5JwD+AeQ4mGNBrdSJ79hM2S5B+NSuY9gdKc
cwAuR0K0tLGPRR9XK474RvJP/KqvuE0ab/jAMZyrTUF16/Hvzucyrb0Z8NOIWi9B
M04hvPD/W9zyUTOlp/DJlJK1RDbvh0STRdvFJg4kz9Q8n7R702Lc2UY9NvRPike2
BHfjZ0p5zbc10hxNFtQJrdybWIckq96G7hc+mB8P36SvSK7In5YZI5MCAwEAAaNG
MEQwEwYDVR0lBAwwCgYIKwYBBQUHAwIwDAYDVR0TAQH/BAIwADAfBgNVHSMEGDAW
gBRFBBI4o0uRg0+Dv6LzwLmyou1PCDANBgkqhkiG9w0BAQsFAAOCAQEAJNE1NuAs
o5cnRU+65ys/+xxBt3Fg8DMj0HokLumvFtZ7CDahX5fPHt6YynAfulgdbhba01UV
c854sOfDO8xYEuaRLCKn+a+yYH7QCOztlJGHlaKiYk7JERdj0u199gpJ+ANLoSQP
fJVwwfclioFf16UDPXNocSJQrjWih34HNrudCy1XjPOu7etjT2ICQ1LD04w77Ls/
speOYBrOQFR33Utn0s/xoGI8ExPzCSpT1Zc/JwRDUoaD6Lu83XaMD24ip+Jj13TY
uI62+u0VfMvp8eiS6MigwP7w7vMD6XqyDF1yXmsnAFspEhuiJcG6fGO2OnkcRb8P
Q5c+ijY0QDP3GA==
-----END CERTIFICATE-----
[root@master1-163 sefe]#
[root@master1-163 sefe]# kubectl create clusterrolebinding test1 --clusterrole=cluster-admin --user=ccx
clusterrolebinding.rbac.authorization.k8s.io/test1 created
[root@master1-163 sefe]# cp /etc/kubernetes/pki/ca.crt .
[root@master1-163 sefe]## 下面地址改为：happroxy的ip
[root@master1-163 sefe]#  kubectl config --kubeconfig=kc1 set-cluster cluster1 --server=https://192.168.59.164:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "cluster1" set.
[root@master1-163 sefe]# kubectl config --kubeconfig=kc1 set-credentials ccx  --client-certificate=ccx.crt --client-key=ccx.key --embed-certs=true
User "ccx" set.
[root@master1-163 sefe]#
[root@master1-163 sefe]#  kubectl config --kubeconfig=kc1 set-context context1 --cluster=cluster1 --namespace=default --user=ccx
Context "context1" created.
[root@master1-163 sefe]# vi kc1
[root@master1-163 sefe]# cat kc1
apiVersion: v1
clusters:
- cluster:certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==server: https://192.168.59.164:6443name: cluster1
contexts:
- context:cluster: cluster1namespace: defaultuser: ccxname: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccxuser:client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@master1-163 sefe]#

至此呢，kubeconfig文件就准备好了，我们重命名为kc2，并拷贝到客户端主机上吧【之前有个kc1，免得冲突】

[root@master1-163 sefe]# mv kc1 kc2
[root@master1-163 sefe]# scp kc2 192.168.59.156:~
The authenticity of host '192.168.59.156 (192.168.59.156)' can't be established.
ECDSA key fingerprint is SHA256:zRtVBoNePoRXh9aA8eppKwwduS9Rjjr/kT5a7zijzjE.
ECDSA key fingerprint is MD5:b8:53:cc:da:86:2a:97:dc:bd:64:6b:b1:d0:f3:02:ce.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.59.156' (ECDSA) to the list of known hosts.
root@192.168.59.156's password:
kc2                                                    100% 5507     4.0MB/s   00:00
[root@master1-163 sefe]#

客户端测试

说明

上面我们已经把kc2文件拷贝到当前服务器上了
并且kc2中连接ip是happroxy的ip地址，你确认下是否有误。

所有配置文件内容如下

[root@etcd1 ~]# cat kc2
apiVersion: v1
clusters:
- cluster:certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM1ekNDQWMrZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJeE1URXlPVEEwTURVME5sb1hEVE14TVRFeU56QTBNRFUwTmxvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBSjBuCm94Ulh2Z2VrUjVkb08xRFRWL0VUYXh3MDE4YndmRC9ZS1lnaUM3OEJxZ1diSUpLaHNrd25LSEtEVkxIcGxvRzkKMEVhcXMxS0UxajgwVW1MVE1pV2lPQS84UG5Kd0dyU3plYjZGRzdpT0VxMVdHTVIzV0VuQW9uaDRESXh4Sjg1SwpyamVDVEU2eHFYdGdydHJ6Z2Y3VWx0TEhsZGtZREswVnc3Y08vZmNoRTN6ZzVJNDR1TWxDeGM4QjVZYUkyMURYCmlrZXFtUlJSQUdYbVpmaEwxYUxzaVEveHFETjgyb0J4bHQ0MzFRbEFDQ2VhQWZlNXhBdlNWYjVsWGRGSWxQZisKeTN6aFAvT3N1VTI3UUlENTVRWVllRmhHUmVtTHlNQWxiOTV4SVRkVkNKbVovRHRyNEF3TUR0ekM5VDhvUFRhcwpyS0l2YVBseHhPai9MM0ZmSXBrQ0F3RUFBYU5DTUVBd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0hRWURWUjBPQkJZRUZFVUVFamlqUzVHRFQ0Ty9vdlBBdWJLaTdVOElNQTBHQ1NxR1NJYjMKRFFFQkN3VUFBNElCQVFBZnN1Yk55M0ZlbzdlNVVYZzIyK2dxaHJsTDVsM245cXgzNlUxVWd3L3VQbnUweXJySwo3clhTSE9MMWFPejRUM29TWVg1VWttTXN0Q1o0RU8ycmFrUVVSVUl4THBzZ3F1OHlDNHRuRldrN1A0UjBiUlU3CmYvWjlETkU3MDR2Rkl2cHhtVGFUdVhzbDhoaVpBUFRGNER3b25hMURXWkx0Y1QxbHcwTEU4TWdEMVFIZG5iSUoKbGxmRlRwZ1RlMS9uQ3BIYWNOeWU2Wk1zVjBPRUovc1RBRXNkSGRwQXQ0VHJoSHZDM1NBeEFrVFJrUENPSGNwWApKbHh3blNYSGF0OGtlRWlQdHdha2E2RW4wQ1B1VUtKTktHVEdXbEUwVmZ1MWZteW5tTW0xMUt4M1N1NHFIMHJhCkh5citmRUlHSE9ib0FpWEVDSXNNSFprN1VlQlYzd3R3OE9DeAotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==server: https://192.168.59.164:6443name: cluster1
contexts:
- context:cluster: cluster1namespace: defaultuser: ccxname: context1
current-context: "context1"
kind: Config
preferences: {}
users:
- name: ccxuser:client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCakNDQWU2Z0F3SUJBZ0lSQUpudElTVWJSRUplcVhEVjd6MitIUVV3RFFZSktvWklodmNOQVFFTEJRQXcKRlRFVE1CRUdBMVVFQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TVRFeE16QXdNalV3TWpkYUZ3MHlNakV4TXpBdwpNalV3TWpkYU1DQXhFREFPQmdOVkJBb1RCMk5yWVRJd01qRXhEREFLQmdOVkJBTVRBMk5qZURDQ0FTSXdEUVlKCktvWklodmNOQVFFQkJRQURnZ0VQQURDQ0FRb0NnZ0VCQU51QzVmVG1MY3d6b1cwMk95Zll0c1ZZeUtnTDMyaXEKV2ZFYW9UWnlaWUEwdjJDRWsxNlRzdWpqU1Mrc2tIKzR3dGJ6allKSlJSWDd5czVOZXU3VUwvcTEzZmxGeVJjSgpIVDJTcG4xWmNzVEdRMWdWMEJ5S0UwcUdmRXFwdjVKd0QrQWVRNG1HTkJyZFNKNzloTTJTNUIrTlN1WTlnZEtjCmN3QXVSMEswdExHUFJSOVhLNDc0UnZKUC9LcXZ1RTBhYi9qQU1aeXJUVUYxNi9Idnp1Y3lyYjBaOE5PSVdpOUIKTTA0aHZQRC9XOXp5VVRPbHAvREpsSksxUkRidmgwU1RSZHZGSmc0a3o5UThuN1I3MDJMYzJVWTlOdlJQaWtlMgpCSGZqWjBwNXpiYzEwaHhORnRRSnJkeWJXSWNrcTk2RzdoYyttQjhQMzZTdlNLN0luNVlaSTVNQ0F3RUFBYU5HCk1FUXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUhBd0l3REFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVcKZ0JSRkJCSTRvMHVSZzArRHY2THp3TG15b3UxUENEQU5CZ2txaGtpRzl3MEJBUXNGQUFPQ0FRRUFKTkUxTnVBcwpvNWNuUlUrNjV5cy8reHhCdDNGZzhETWowSG9rTHVtdkZ0WjdDRGFoWDVmUEh0Nll5bkFmdWxnZGJoYmEwMVVWCmM4NTRzT2ZETzh4WUV1YVJMQ0tuK2EreVlIN1FDT3p0bEpHSGxhS2lZazdKRVJkajB1MTk5Z3BKK0FOTG9TUVAKZkpWd3dmY2xpb0ZmMTZVRFBYTm9jU0pRcmpXaWgzNEhOcnVkQ3kxWGpQT3U3ZXRqVDJJQ1ExTEQwNHc3N0xzLwpzcGVPWUJyT1FGUjMzVXRuMHMveG9HSThFeFB6Q1NwVDFaYy9Kd1JEVW9hRDZMdTgzWGFNRDI0aXArSmoxM1RZCnVJNjIrdTBWZk12cDhlaVM2TWlnd1A3dzd2TUQ2WHF5REYxeVhtc25BRnNwRWh1aUpjRzZmR08yT25rY1JiOFAKUTVjK2lqWTBRRFAzR0E9PQotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBMjRMbDlPWXR6RE9oYlRZN0o5aTJ4VmpJcUF2ZmFLcFo4UnFoTm5KbGdEUy9ZSVNUClhwT3k2T05KTDZ5UWY3akMxdk9OZ2tsRkZmdkt6azE2N3RRdityWGQrVVhKRndrZFBaS21mVmx5eE1aRFdCWFEKSElvVFNvWjhTcW0va25BUDRCNURpWVkwR3QxSW52MkV6WkxrSDQxSzVqMkIwcHh6QUM1SFFyUzBzWTlGSDFjcgpqdmhHOGsvOHFxKzRUUnB2K01BeG5LdE5RWFhyOGUvTzV6S3R2Um53MDRoYUwwRXpUaUc4OFA5YjNQSlJNNlduCjhNbVVrclZFTnUrSFJKTkYyOFVtRGlUUDFEeWZ0SHZUWXR6WlJqMDI5RStLUjdZRWQrTm5Tbm5OdHpYU0hFMFcKMUFtdDNKdFloeVNyM29idUZ6NllIdy9mcEs5SXJzaWZsaGtqa3dJREFRQUJBb0lCQVFDaFd2RUtPZ0RFTDllagpYYy9TRkgwVlI1UUg0dUpRSDVpSm9GZU1uRDU4SlVuZit0UVJHMlRSeC9ET09Iem5SYnNESW5pTW9xdEQ0NWhLCldhM1p6T09QMlF2WDVqSlEyb1JCOUlDcGQ0empsQkdBdUZnSUFuNzNzeSs5K2xVMW9XWXFDbFQrekVXVTBjQkcKRG5rR1c4bVFYOTRFcklXM2VRVVh4dXplM2RKTkNYUTdsYmExM0dqRnJkeWNncTd1SktDVUlOZVhrYnVUU09pWgpHeUhwSC9wYm5tdzNxV1lWcXJCMHZydUNVWEV2NkpFUU9wZjVrQmVSRVVGV2owbTB4MFcvUGJOczJUWWRRNDZSCndwVUdzemd6UFE3aTd0R1Q1bVJVeXljem92clRJSzRyZVMyb2VJY0lvdHo3dk5aUFBMMFdDWGVuTlYyUVh0M0YKVEJpK2FJcEJBb0dCQVB6a29VZE1nNXpLUzhEV3VDUUhMaXlCYVpqdnVnR3dvZ2svcFRDc2ZUZlRuYnZISHE3cgpENVF1aXI1Ym92UTRPeDZnUWFpcjJpWURJU0JXQ3grQXFNYnB3cnV2TkpjYWJiWDg3bG0xOThnOUhQU0FBeHZYCldMWmVJRUovZ1lZeXEydGwxU2x3aXlpcUJVKzk1TFdPWkNSMWdQemF4cTY3aHNHZjNCUVh0c0hEQW9HQkFONDEKUjZJOVpzSjcwT2lyNUdORExDMWw2L3lzYk9UMHpkUC9abXA0QzE3ZlZNY1czTDNBOFZ6NkU2bEVDajNJV21NeAovRXdjSWl6Y3RwMmRPdXEvQXQvQkhMK2R3Z2JOaStTT0s5SGNZVWlRczlkZGZmemFLRG9YbU0wZC9Ec0hjemJmCjYvTGRMM2lJaHBmNTBrZU4yOG81RE8vRWVpUGh6OTNyMUtHaElhbnhBb0dCQUtUUitZNzNmanU4Mk54bzFRQ2UKTVdqT08xMXl1RjhMbUwvQVhGQTV5eTZNWEJ3YkxaTkJIaGtzc0Q1YWlQejBmUXQ3MU84eWVlNFdVZ3U2S0d1eQp0YzFXWXhWaG5qdncvSm9FcSsrS0dtREYrODRhZEd3NzBOU2l5aEdzK21UVk4wVzZ5OC9EU3Q4STJJZGRNdWRsClV6MHozQXB3SjllUDNzYnBBazJTR3dsekFvR0FKWkZNY3hsK2JoTVExc1Y0NzI5RDRNa0NoTTJCKzVPSDJQZXQKRVFNS3FSZUk5Yi9md2hVRXVYdHVKNGZoVTJDMEoyRWtEcG9URFJLanJUSVA2L0F2UkRVWjd5dDB5a0dtRFJZYwpRbmJIMjhUYkl1WWpqc1F6V043MGJubExVNCtHNHhnb2cvN3hMNmVsV2J4YTJNNGJBcTF5aW5ibFQzdTE4UWNXCkE5MkNHNEVDZ1lCemVHM2d6akNSWVN3b3ZENXd3WWR6Yi9yRTM5RXpuS2lPVm9xc003MHhWaVhBbjF5NlBGcisKZXBudVA2Mzk1VC8zQzFKUnNJU1cydEdIaFAyN0ZKUHVGKytTZWFheUMvdUZTZEkvWEpLVkx6Y2FrMmoyMHdWOApBTWJVeTNEWlQ2c0JDU2Jhclp3cEM2VUFUUzZHUzQxOWRmWTdJZzdzSHhKM3R2VWNTUUx1N3c9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@etcd1 ~]#

为什么要连接到happroxy，因为我们在happroxy上配置了，只要访问当前ip的6443端口，就会自动转发到master上
我们在happroxy上查看规则

[root@haproxy-164 haproxy]# netstat -ntlp | grep 6443
tcp        0      0 0.0.0.0:6443            0.0.0.0:*               LISTEN      2504/haproxy
[root@haproxy-164 haproxy]# tail /etc/haproxy/haproxy.cfgserver  app1 127.0.0.1:5001 checkserver  app2 127.0.0.1:5002 checkserver  app3 127.0.0.1:5003 checkserver  app4 127.0.0.1:5004 checklisten k8s-lb *:6443mode tcpbalance roundrobinserver s1 192.168.59.163:6443 weight 1server s2 192.168.59.162:6443 weight 1
[root@haproxy-164 haproxy]#

测试

如下，没有kubectl命令的话，需要先安装一个命令哈

[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get nodes
-bash: kubectl: 未找到命令
[root@etcd1 ~]#
# 我是离线安装的
[root@etcd1 kubelet]# rpm -ivhU * --nodeps --force
警告：3944a45bec4c99d3489993e3642b63972b62ed0a4ccb04cc7655ce0467fddfef-kubectl-1.21.1-0.x86_64.rpm: 头V4 RSA/SHA512 Signature, 密钥 ID 3e1ba8d5: NOKEY
准备中...                          ################################# [100%]
正在升级/安装...1:socat-1.7.3.2-2.el7              ################################# [  9%]2:libnetfilter_queue-1.0.2-2.el7_2 ################################# [ 18%]3:libnetfilter_cttimeout-1.0.0-7.el################################# [ 27%]4:libnetfilter_cthelper-1.0.0-11.el################################# [ 36%]5:libnetfilter_conntrack-1.0.6-1.el################################# [ 45%]6:conntrack-tools-1.4.4-7.el7      ################################# [ 55%]7:kubernetes-cni-0.8.7-0           ################################# [ 64%]8:kubelet-1.21.1-0                 ################################# [ 73%]9:cri-tools-1.19.0-0               ################################# [ 82%]10:kubectl-1.21.1-0                 ################################# [ 91%]11:kubeadm-1.21.1-0                 ################################# [100%]
[root@etcd1 kubelet]# systemctl enable kubelet.service --now
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@etcd1 kubelet]#

再来，测试
没问题，完美啊，卧槽~ 一次性成功了这次没有处理报错好感动啊

[root@etcd1 kubelet]# cd
[root@etcd1 ~]# export KUBECONFIG=kc2
[root@etcd1 ~]# kubectl get nodes
NAME          STATUS   ROLES                  AGE   VERSION
master1-163   Ready    control-plane,master   23h   v1.21.1
master2-162   Ready    control-plane,master   22h   v1.21.1
worker-165    Ready    <none>                 18h   v1.21.1
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME              STATUS   AGE
default           Active   23h
kube-node-lease   Active   23h
kube-public       Active   23h
kube-system       Active   23h
ns1               Active   100m
ns2               Active   99m
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl create ns ns3
namespace/ns3 created
[root@etcd1 ~]#
[root@etcd1 ~]# kubectl get ns
NAME              STATUS   AGE
default           Active   23h
kube-node-lease   Active   23h
kube-public       Active   23h
kube-system       Active   23h
ns1               Active   100m
ns2               Active   99m
ns3               Active   6s
[root@etcd1 ~]#

更好的部署master高可用的方式

这是国内的一个方式，叫做sealos收费的，但是一年就几十块钱，很便宜，而且一件部署，炒鸡方便啊~ 上面手动部署的太麻烦了，如果不想折腾的，那么sealos绝对是你的首选哦
官网：
kubernetes 集群离线安装包
一条命令高可用支持国产化
官网已经做了很详细的使用流程了，而且很简单
我是干技术的，所以呢我还是喜欢使用上面自己动手的配置方法，我不会使用这个，所以我这就不多累赘了，感兴趣的小伙伴去官网注册试试把 ~

【kubernetes】k8s使用客户端连接haproxy访问高可用集群流程详细说明【使用kubeconfig连接haproxy】【kubeconfig配置全部流程】相关推荐

s19.基于 Kubernetes v1.25 (kubeadm) 和 Docker 部署高可用集群(一)
基于 Kubernetes v1.25 和 Docker 部署高可用集群主要内容 Kubernetes 集群架构组成容器运行时 CRI Kubernetes v1.25 新特性 Kubernete ...
Haproxy+keepalived高可用集群实战
前言随着互联网火热的发展,开源负载均衡器的大量的应用,企业主流软件负载均衡如LVS.Haproxy.Nginx等,各方面性能不亚于硬件负载均衡F5,Haproxy提供高可用性.负载均衡以及基于TCP ...
s24.基于 Kubernetes v1.25 (二进制) 和 Docker部署高可用集群
1.安装说明本文章将演示二进制方式安装高可用k8s 1.17+,相对于其他版本,二进制安装方式并无太大区别,只需要区分每个组件版本的对应关系即可. 生产环境中,建议使用小版本大于5的Kubernet ...
s20.基于 Kubernetes v1.25 (kubeadm) 和 Docker 部署高可用集群(二)
4.4 安装 Docker master和node安装docker-ce: [root@k8s-master01 ~]# cat install_docker.sh #!/bin/bash # #** ...
Kubernetes — 在 OpenStack 上使用 kubeadm 部署高可用集群
目录文章目录目录高可用集群部署拓扑高可用集群网络拓扑网络代理配置 Load Balancer 环境准备 Kubernetes Cluster 环境准备安装 Container Runtim ...
【❤️万字长文总结❤️】一篇学会Redis高可用✔集群✔搭建详细教程
大家好,我是Lex 喜欢欺负超人那个Lex 擅长领域:python开发.网络安全渗透.Windows域控Exchange架构今日重点:今天总结一下Redis集群高可用的搭建流程 [惊喜推荐+优质资源 ...
【Docker Swarm】搭建Docker Swarm高可用集群（详细版）
文章目录 1️⃣什么是Docker Swarm? 2️⃣Docker Swarm相关命令说明 3️⃣Docker Swarm集群节点规划 4️⃣创建Docker Swarm高可用集群操作步骤演示
Haproxy+Heartbeat 高可用集群方案操作记录
之前详细介绍了haproxy的基础知识点, 下面记录下Haproxy+Heartbeat高可用web集群方案实现过程, 以加深理解. 架构草图如下: 1) 基本环境准备 (centos6.9系统) 1 ...
centos7 安装haproxy+rabbitmq高可用集群
一,准备工作: 1,三台centos7虚拟机: 192.168.209.143 rabbitmq-node1+haproxy 192.168.209.147 rabbitmq-node2 192. ...

【kubernetes】k8s使用客户端连接haproxy访问高可用集群流程详细说明【使用kubeconfig连接haproxy】【kubeconfig配置全部流程】

文章目录

master高可用部署流程

客户端连接haproxy访问高可用集群

环境确认与准备【必看】

客户端连接happroxy说明

kubeconfig配置【master上操作】

客户端测试

说明

测试

更好的部署master高可用的方式

【kubernetes】k8s使用客户端连接haproxy访问高可用集群流程详细说明【使用kubeconfig连接haproxy】【kubeconfig配置全部流程】相关推荐

最新文章

热门文章