继续前一章的部署。

十、kube-scheduler集群部署

该集群包含2个节点，启动后通过竞争选举机制产生一个leader节点，其他节点为阻塞状态。当leader节点不可用后，剩余的节点将再次进行选举产生新的leader节点，从而保证服务的可用性。

如下两种情况下使用x509证书：

a. 与 kube-apiserver 的安全端口通信

b. 在安全端口（https，10251）输出 prometheus 格式的 metrics

10.1 下载二进制文件，参考 第三章

10.2 创建 kube-scheduler 证书和私钥

创建证书签名请求：

cat > kube-scheduler-csr.json <<EOF
{"CN": "system:kube-scheduler","hosts": ["127.0.0.1","192.168.56.20","192.168.56.21"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "BeiJing","L": "BeiJing","O": "system:kube-scheduler","OU": "4Paradigm"}]
}
EOF

• hosts 列表包含所有 kube-scheduler 节点IP
• CN为 system:kube-scheduler， O 为 system:kube-scheduler ， kubernetes 内置的 ClusterRoleBindings system:kube-scheduler 将赋予 kube-scheduler 工作所需的权限

生成证书和私钥：

cfssl gencert -ca=/etc/kubernetes/cert/ca.pem \-ca-key=/etc/kubernetes/cert/ca-key.pem \-config=/etc/kubernetes/cert/ca-config.json \-profile=kubernetes kube-scheduler-csr.json | cfssljson -bare kube-scheduler

10.3 创建和分发 kubeconfig 文件

kubeconfig 文件包含访问 apiserver 的所有信息，如 apiserver地址，CA证书和自身使用的证书

source /opt/k8s/bin/environment.shkubectl config set-cluster kubernetes \--certificate-authority=/etc/kubernetes/cert/ca.pem \--embed-certs=true\--server=${KUBE_APISERVER} \--kubeconfig=kube-scheduler.kubeconfigkubectl config set-credentials system:kube-scheduler \--client-certificate=kube-scheduler.pem \--client-key=kube-scheduler-key.pem \--embed-certs=true\--kubeconfig=kube-scheduler.kubeconfigkubectl config set-context system:kube-scheduler \--cluster=kubernetes \--user=system:kube-scheduler \--kubeconfig=kube-scheduler.kubeconfigkubectl config use-context system:kube-scheduler --kubeconfig=kube-scheduler.kubeconfig

• 上一步创建的证书、私钥以及 kube-apiserver 地址被写入到 kubeconfig 文件中

分发 kubeconfig 到所有 master 节点

source /opt/k8s/bin/environment.sh
for master_ip in${MASTER_IPS[@]}doecho ">>> ${master_ip}"scp kube-scheduler.kubeconfig k8s@${master_ip}:/etc/kubernetes/
done

10.4 创建和分发 kube-scheduler systemd unit 文件

cat > kube-scheduler.service <<EOF
[Unit]
Description=Kubernetes Scheduler
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
[Service]
ExecStart=/opt/k8s/bin/kube-scheduler \\--address=127.0.0.1\\--kubeconfig=/etc/kubernetes/kube-scheduler.kubeconfig \\--leader-elect=true\\--alsologtostderr=true\\--logtostderr=false\\--log-dir=/var/log/kubernetes \\--v=2Restart=on-failure
RestartSec=5User=k8s[Install]
WantedBy=multi-user.target
EOF

• --address 在 127.0.0.1:10251 端口接收 http /metrics 请求； kube-scheduler 目前还不支持接收 https 请求
• --kubeconfig 指定 kubeconfig 文件路径，kube-scheduler 使用它链接和验证 kube-apiserver
• --leader-elect=true 集群运行模式，启用选举功能； 被选为 leader 的节点负责处理工作，其他节点为阻塞状态
• User=k8s  使用 k8s 账户运行

分发 systemd unit 文件到所有 master 节点：

source /opt/k8s/bin/environment.sh
for master_ip in${MASTER_IPS[@]}doecho ">>> ${master_ip}"scp kube-scheduler.service root@${master_ip}:/etc/systemd/system/
done

10.5 启动 kube-scheduler 服务

source /opt/k8s/bin/environment.sh
for master_ip in${MASTER_IPS[@]}doecho ">>> ${master_ip}"ssh root@${master_ip} "mkdir -p /var/log/kubernetes && chown -R k8s /var/log/kubernetes"ssh root@${master_ip} "systemctl daemon-reload && systemctl enable kube-scheduler && systemctl restart kube-scheduler"
done

• 运行前，必须先创建日志目录

10.6 检查服务运行状态

source /opt/k8s/bin/environment.sh
for master_ip in${MASTER_IPS[@]}doecho ">>> ${master_ip}"ssh k8s@${master_ip} "systemctl status kube-scheduler|grep Active"
done

• 确认服务是 Active: active (running) 状态
• 查看日志： journalctl -u kube-scheduler

10.7 查看输出的 metric

以下命令在 kube-scheduler 节点上执行

[root@k8s-m1 template]# sudo netstat -lnpt|grep kube-sche
tcp0      0 127.0.0.1:10251         0.0.0.0:*               LISTEN      6800/kube-scheduler[root@k8s-m1 template]# curl -s http://127.0.0.1:10251/metrics |head
# HELP apiserver_audit_event_total Counter of audit events generated and sent to the audit backend.
# TYPE apiserver_audit_event_total counter
apiserver_audit_event_total0# HELP apiserver_client_certificate_expiration_seconds Distribution of the remaining lifetime on the certificate used to authenticate a request.
# TYPE apiserver_client_certificate_expiration_seconds histogram
apiserver_client_certificate_expiration_seconds_bucket{le="0"} 0apiserver_client_certificate_expiration_seconds_bucket{le="21600"} 0apiserver_client_certificate_expiration_seconds_bucket{le="43200"} 0apiserver_client_certificate_expiration_seconds_bucket{le="86400"} 0apiserver_client_certificate_expiration_seconds_bucket{le="172800"} 0

10.8 测试 kube-scheduler 集群的高可用

找到一个节点，停掉服务，查看其它节点是否获取了leader权限

10.9 查看当前的 leader

kubectl get endpoints kube-scheduler --namespace=kube-system  -o yaml