k8s etcd-2 Unhealthy HTTP probe failed with statuscode: 503

[root@vlin121 bin]# kubectl get componentstatuses --server=192.168.133.121:8080 k8s验证错误
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-2 Unhealthy HTTP probe failed with statuscode: 503
etcd-0 Healthy {"health":"true"}
etcd-1 Healthy {"health":"true"}

cat /var/log/messge

Aug 25 18:37:21 vlin121 etcd: health check for peer 5e2c50125f89e6fc could not connect: dial tcp 1
92.168.133.122:2380: connect: connection refused (prober "ROUND_TRIPPER_SNAPSHOT")Aug 25 18:37:21 vlin121 etcd: health check for peer ad0d52b54b7da12 could not connect: dial tcp 19
2.168.133.123:2380: connect: connection refused (prober "ROUND_TRIPPER_RAFT_MESSAGE")Aug 25 18:37:21 vlin121 etcd: health check for peer 5e2c50125f89e6fc could not connect: dial tcp 1
92.168.133.122:2380: connect: connection refused (prober "ROUND_TRIPPER_RAFT_MESSAGE")Aug 25 18:37:21 vlin121 etcd: health check for peer ad0d52b54b7da12 could not connect: dial tcp 19
2.168.133.123:2380: connect: connection refused (prober "ROUND_TRIPPER_SNAPSHOT")Aug 25 18:37:22 vlin121 etcd: e8c545a5e7757ebe [logterm: 250, index: 318, vote: 5e2c50125f89e6fc]
ignored MsgVote from ad0d52b54b7da12 [logterm: 249, index: 14] at term 250: lease is not expired (remaining ticks: 10)

[root@vlin121 bin]# etcdctl --endpoints=http://192.168.133.121:2379 cluster-health 验证etcd有个节点出现错误
member ad0d52b54b7da12 is unhealthy: got unhealthy result from http://192.168.133.123:2379
member 5e2c50125f89e6fc is healthy: got healthy result from http://192.168.133.122:2379
member e8c545a5e7757ebe is healthy: got healthy result from http://192.168.133.121:2379

[root@vlin121 bin]# scp /etc/etcd/etcd.conf 192.168.133.123:/etc/etcd/

[root@vlin123 docker]# vim /etc/etcd/etcd.conf 上面四处加主机名改本机的

ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
#ETCD_WAL_DIR=""
ETCD_LISTEN_PEER_URLS="http://192.168.133.123:2380"
ETCD_LISTEN_CLIENT_URLS="http://192.168.133.123:2379"
#ETCD_MAX_SNAPSHOTS="5"
#ETCD_MAX_WALS="5"
ETCD_NAME="vlin123.ywz.com"
#ETCD_SNAPSHOT_COUNT="100000"
#ETCD_HEARTBEAT_INTERVAL="100"
#ETCD_ELECTION_TIMEOUT="1000"
#ETCD_QUOTA_BACKEND_BYTES="0"
#ETCD_MAX_REQUEST_BYTES="1572864"
#ETCD_GRPC_KEEPALIVE_MIN_TIME="5s"
#ETCD_GRPC_KEEPALIVE_INTERVAL="2h0m0s"
#ETCD_GRPC_KEEPALIVE_TIMEOUT="20s"
#
#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://192.168.133.123:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://192.168.133.123:2379"
#ETCD_DISCOVERY=""
#ETCD_DISCOVERY_FALLBACK="proxy"
#ETCD_DISCOVERY_PROXY=""
#ETCD_DISCOVERY_SRV=""
ETCD_INITIAL_CLUSTER="vlin121.ywz.com=http://192.168.133.121:2380,vlin122.ywz.com=http://192.168.133.122:238
0,vlin123.ywz.com=http://192.168.133.123:2380"ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"

[root@vlin123 docker]# systemctl restart etcd.service ; systemctl status etcd.service

[root@vlin121 bin]# etcdctl --endpoints=http://192.168.133.121:2379 cluster-health
member ad0d52b54b7da12 is healthy: got healthy result from http://192.168.133.123:2379
member 5e2c50125f89e6fc is healthy: got healthy result from http://192.168.133.122:2379
member e8c545a5e7757ebe is healthy: got healthy result from http://192.168.133.121:2379

至此全好了

[root@vlin121 bin]# kubectl get componentstatuses --server=192.168.133.121:8080
NAME STATUS MESSAGE ERROR
controller-manager Healthy ok
scheduler Healthy ok
etcd-1 Healthy {"health":"true"}
etcd-2 Healthy {"health":"true"}
etcd-0 Healthy {"health":"true"}

k8s etcd-2 Unhealthy HTTP probe failed with statuscode: 503相关推荐

k8s报错503或者其他网络错误 Readiness probe failed: HTTP probe failed with statuscode: 503
【解决方案】istio-ingressgateway HTTP probe failed with statuscode: 503
简单描述直观上来看,问题是出在istio-proxy这个sidecar容器上了. 一般来说,问题是处在了istio-system中的pilot和容器之间的连接上了.这样看来,其实问题有很多种,比如 ...
k8s etcd 连接错误
今天在用代码连接k8s内部的etcd的时候,遇到了下面这个错误,全网搜索也没有相似的: {"level":"warn","ts":" ...
从零开始入门 K8s | etcd 性能优化实践
作者 | 陈星宇(宇慕) 阿里云基础技术中台技术专家本文整理自<CNCF x Alibaba 云原生技术公开课>第 17 讲. 导读:etcd 是容器云平台用于存储关键元信息的组件.阿里 ...
解决K8S集群kubelet故障：Failed to update stats for container nable to determine device info for dir
在检查k8S节点系统日志时发现,有大量的kubelet组件报错,报错内容如下: Jun 16 09:51:37 master kubelet: W0616 09:51:37.076817 6997 c ...
k8s中pod sandbox创建失败failed to start sandbox container
背景今天在k8s更新服务时,发现pod启动失败,报错failed to start sandbox container,如下所示: Events:Type Reason Age From Messa ...
k8s, etcd 多节点集群部署问题排查记录
目录文章目录目录部署环境 1. etcd 集群启动失败解决 2. etcd 健康状态检查失败解决 3. kube-apiserver 启动失败解决 4. kubelet 启动失败解决 5 ...
Liveness probe failed: OCI runtime exec failed: exec failed: container_linux.go:345:starting
k8s容器配置liveness探针,启动之后报错如下查看应用的liveness配置如下: 排查发现启动命令没有指定shell,修改如下: 加上sh -c之后,重启pod之后正常.
【网址收藏】k8s HPA自动伸缩异常：failed to get cpu utilization: missing request
参考链接: Kubernetes:HPA 详解-基于 CPU.内存和自定义指标自动扩缩容

k8s etcd-2 Unhealthy HTTP probe failed with statuscode: 503

k8s etcd-2 Unhealthy HTTP probe failed with statuscode: 503相关推荐

最新文章

热门文章