1、KubeSphere

https://www.yuque.com/leifengyang/oncloud/gz1sls

介绍

sphere
英
/sfɪə(r)/
n.
（活动、兴趣、专业知识的）范围，领域；球体，球，球形；社会阶层；势力范围；<文>苍穹，天，天空；<文>天体；天体外壳；地球仪
vt.
使……成球形；包围；放入球内

自动化运维部署
应用的监控
日志收集
系统告警
官网
https://kubesphere.com.cn/
青云公司的网址，无用。
- https://kubesphere.qingcloud.com/

KubeSphere是基于Kubernetes构建的分布式、多租户、多集群、企业级开源容器平台，具有强大旦完善的网络与存储能力，并通过极简的人机交互提供完善的多集群管理、CI/CD、微服务治理、应用管理等功能，帮助企业在云、虚拟化及物理机等异构基础设施上快速
构建、部署及运维容器架构，实现应用的敏捷开发与全生命周期管理。

遵循Apache 2.0 协议的。
KubeSphere 是容器平台。k8s是容器编排系统。
多租户
- 用户可以自己注册
- 可以分配权限。
多集群
- 开发和测试的集群。

integration
n.
结合，融合；取消种族隔离；（数）积分法，求积分；（心理分析）整合

最新为：KubeSphere 企业版 3.3.1 正式上线 ➝

从 v3.3.1 开始， QingCloud KubeSphere Container Platform （QKCP）更名为 KubeSphere 企业版（KubeSphere Enterprise）。

青云旗下有一个开源软件叫KubeSphere，它是国内唯一一个Kubernetes发行版，与Rancher和OpenShift一样的发行版，就好像RHEL、Ubuntu是Linux内核的发行版一样，发行版让用户以更友好的方式使用Linux操作系统的各种功能，而自己默默担下了很多重要工作。

2、docker 和 k8s安装

Kubernetes上安装KubeSphere

安装步骤

选择4核8G（master）、8核16G（node1）、8核16G（node2）三台机器，按量付费进行实验，CentOS7.9
安装Docker
安装Kubernetes
安装KubeSphere前置环境
安装KubeSphere

1、安装Docker

sudo yum remove docker*
sudo yum install -y yum-utils#配置docker的yum地址
sudo yum-config-manager \
--add-repo \
http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo#安装指定版本
sudo yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io-1.4.6#    启动&开机启动docker
systemctl enable docker --now# docker加速配置
sudo mkdir -p /etc/docker
sudo tee /etc/docker/daemon.json <<-'EOF'
{"registry-mirrors": ["https://82m9ar63.mirror.aliyuncs.com"],"exec-opts": ["native.cgroupdriver=systemd"],"log-driver": "json-file","log-opts": {"max-size": "100m"},"storage-driver": "overlay2"
}
EOF
sudo systemctl daemon-reload
sudo systemctl restart docker

2、安装Kubernetes

1、基本环境

每个机器使用内网ip互通

每个机器配置自己的hostname，不能用localhost

#设置每个机器自己的hostname。一个master 两个node
hostnamectl set-hostname xxx# 将 SELinux 设置为 permissive 模式（相当于将其禁用）
sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config#关闭swap
swapoff -a
sed -ri 's/.*swap.*/#&/' /etc/fstab#允许 iptables 检查桥接流量
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOFcat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

2、安装kubelet、kubeadm、kubectl

#配置k8s的yum源地址
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttp://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF#安装 kubelet，kubeadm，kubectl
sudo yum install -y kubelet-1.20.9 kubeadm-1.20.9 kubectl-1.20.9#启动kubelet
sudo systemctl enable --now kubelet#所有机器配置master域名
echo "172.31.0.4  k8s-master" >> /etc/hosts

3、初始化master节点

kubeadm init \
--apiserver-advertise-address=172.31.0.4 \
--control-plane-endpoint=k8s-master \
--image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images \
--kubernetes-version v1.20.9 \
--service-cidr=10.96.0.0/16 \
--pod-network-cidr=192.168.0.0/16

记录关键信息

记录master执行完成后的日志

mkdir -p $HOME/.kubesudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/configsudo chown $(id -u):$(id -g) $HOME/.kube/configkubeadm join k8s-master:6443 --token 3vckmv.lvrl05xpyftbs177 \--discovery-token-ca-cert-hash sha256:1dc274fed24778f5c284229d9fcba44a5df11efba018f9664cf5e8ff77907240 \--control-plane #不加这个，是加入node

4、安装Calico网络插件

curl https://docs.projectcalico.org/manifests/calico.yaml -O
# 注意，这个网络插件不能用。换更高版本的 镜像无法下载。
# 需要手动 把镜像 导入到 本地的 dockerkubectl apply -f calico.yaml

最后：加入worker节点

3、KubeSphere前置安装

https://kubesphere.io/zh/docs/v3.3/quick-start/minimal-kubesphere-on-k8s/

前置环境 nfs

在安装之前，需要配置Kubernetes集群中的默认存储类型。

1、nfs文件系统

1、安装nfs-server

# 在每个机器。
yum install -y nfs-utils# 在master 执行以下命令
echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports# 执行以下命令，启动 nfs 服务;创建共享目录
mkdir -p /nfs/data# 在master执行
systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-server# 使配置生效
exportfs -r#检查配置是否生效
exportfs

2、配置nfs-client（选做）

showmount -e 172.31.0.10mkdir -p /nfs/datamount -t nfs 172.31.0.10:/nfs/data /nfs/data

3、配置默认存储

配置动态供应的默认存储类

## 创建了一个存储类
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: nfs-storageannotations:storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:archiveOnDelete: "true"  ## 删除pv的时候，pv的内容是否要备份---
apiVersion: apps/v1
kind: Deployment
metadata:name: nfs-client-provisionerlabels:app: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
spec:replicas: 1strategy:type: Recreateselector:matchLabels:app: nfs-client-provisionertemplate:metadata:labels:app: nfs-client-provisionerspec:serviceAccountName: nfs-client-provisionercontainers:- name: nfs-client-provisionerimage: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/nfs-subdir-external-provisioner:v4.0.2# resources:#    limits:#      cpu: 10m#    requests:#      cpu: 10mvolumeMounts:- name: nfs-client-rootmountPath: /persistentvolumesenv:- name: PROVISIONER_NAMEvalue: k8s-sigs.io/nfs-subdir-external-provisioner- name: NFS_SERVERvalue: 172.31.0.10 ## 指定自己nfs服务器地址- name: NFS_PATH  value: /nfs/data  ## nfs服务器共享的目录volumes:- name: nfs-client-rootnfs:server: 172.31.0.10path: /nfs/data
---
apiVersion: v1
kind: ServiceAccount
metadata:name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: nfs-client-provisioner-runner
rules:- apiGroups: [""]resources: ["nodes"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["persistentvolumes"]verbs: ["get", "list", "watch", "create", "delete"]- apiGroups: [""]resources: ["persistentvolumeclaims"]verbs: ["get", "list", "watch", "update"]- apiGroups: ["storage.k8s.io"]resources: ["storageclasses"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["events"]verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: run-nfs-client-provisioner
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: ClusterRolename: nfs-client-provisioner-runnerapiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
rules:- apiGroups: [""]resources: ["endpoints"]verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: Rolename: leader-locking-nfs-client-provisionerapiGroup: rbac.authorization.k8s.io

4、查看和测试

 kubectl get sc #查看存储类kubectl get storageclasskubectl get pod -A #动态资源创建，需要这个podnfs-client-provisioner-5bdd4f8d7b-vlzjz

provisioner
英
/prəˈvɪʒənə(r)/
n.
粮食供应者
杂货商

创建 PVC

kind: PersistentVolumeClaim
apiVersion: v1
metadata:name: nginx-pvc
spec:accessModes:- ReadWriteManyresources:requests:storage: 200MistorageClassName: nfs #分组，不用写。#现在默认的为 nfs-storage (default) ，也可写默认的。

kubectl get pv #在查看，发现一个被绑定
pvc-1afe14c8-412b-4379-a14a-834339670246   15Mi       RWX            Delete           Bound

前置环境集群监控

metrics-server

metrics
英
/ˈmetrɪks/
n.
度量；作诗法；韵律学

yaml

集群指标监控组件

apiVersion: v1
kind: ServiceAccount
metadata:labels:k8s-app: metrics-servername: metrics-servernamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:k8s-app: metrics-serverrbac.authorization.k8s.io/aggregate-to-admin: "true"rbac.authorization.k8s.io/aggregate-to-edit: "true"rbac.authorization.k8s.io/aggregate-to-view: "true"name: system:aggregated-metrics-reader
rules:
- apiGroups:- metrics.k8s.ioresources:- pods- nodesverbs:- get- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:labels:k8s-app: metrics-servername: system:metrics-server
rules:
- apiGroups:- ""resources:- pods- nodes- nodes/stats- namespaces- configmapsverbs:- get- list- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:labels:k8s-app: metrics-servername: metrics-server-auth-readernamespace: kube-system
roleRef:apiGroup: rbac.authorization.k8s.iokind: Rolename: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccountname: metrics-servernamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:k8s-app: metrics-servername: metrics-server:system:auth-delegator
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:auth-delegator
subjects:
- kind: ServiceAccountname: metrics-servernamespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:labels:k8s-app: metrics-servername: system:metrics-server
roleRef:apiGroup: rbac.authorization.k8s.iokind: ClusterRolename: system:metrics-server
subjects:
- kind: ServiceAccountname: metrics-servernamespace: kube-system
---
apiVersion: v1
kind: Service
metadata:labels:k8s-app: metrics-servername: metrics-servernamespace: kube-system
spec:ports:- name: httpsport: 443protocol: TCPtargetPort: httpsselector:k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:labels:k8s-app: metrics-servername: metrics-servernamespace: kube-system
spec:selector:matchLabels:k8s-app: metrics-serverstrategy:rollingUpdate:maxUnavailable: 0template:metadata:labels:k8s-app: metrics-serverspec:containers:- args:- --cert-dir=/tmp- --kubelet-insecure-tls- --secure-port=4443- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname- --kubelet-use-node-status-portimage: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/metrics-server:v0.4.3imagePullPolicy: IfNotPresentlivenessProbe:failureThreshold: 3httpGet:path: /livezport: httpsscheme: HTTPSperiodSeconds: 10name: metrics-serverports:- containerPort: 4443name: httpsprotocol: TCPreadinessProbe:failureThreshold: 3httpGet:path: /readyzport: httpsscheme: HTTPSperiodSeconds: 10securityContext:readOnlyRootFilesystem: truerunAsNonRoot: truerunAsUser: 1000volumeMounts:- mountPath: /tmpname: tmp-dirnodeSelector:kubernetes.io/os: linuxpriorityClassName: system-cluster-criticalserviceAccountName: metrics-servervolumes:- emptyDir: {}name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:labels:k8s-app: metrics-servername: v1beta1.metrics.k8s.io
spec:group: metrics.k8s.iogroupPriorityMinimum: 100insecureSkipTLSVerify: trueservice:name: metrics-servernamespace: kube-systemversion: v1beta1versionPriority: 100

测试查看内存占用

kubectl top nodes
NAME       CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
master01   273m         6%     1584Mi          43%# CPU 1核心为1000Mkubectl top pods -Afree -m #查看内存df -h #查看磁盘 disk

4、KubeSphere 安装

v3.3安装：

https://kubesphere.io/zh/docs/v3.3/quick-start/minimal-kubesphere-on-k8s/

您的 Kubernetes 版本必须为：v1.19.x、v1.20.x、v1.21.x、* v1.22.x、* v1.23.x 和 * v1.24.x。带星号的版本可能出现边缘节点部分功能不可用的情况。

下载核心文件

如果下载不到，请复制附录的内容

yum install -y wgetwget https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/kubesphere-installer.yamlwget https://github.com/kubesphere/ks-installer/releases/download/v3.1.1/cluster-configuration.yaml

修改cluster-configuration

在 cluster-configuration.yaml中指定我们需要开启的功能

参照官网“启用可插拔组件”

https://kubesphere.com.cn/docs/pluggable-components/overview/

spec:persistence:storageClass: ""        # If there is no default StorageClass in your cluster, you need to specify an existing StorageClass here. 这个不用改，我们有默认的。etcd:monitoring: false #etcd监控改为trueendpointIps: localhost  # etcd cluster EndpointIps. It can be a bunch of IPs here. 改为masterIP：172.31.0.10common:redis:enabled: false #redis开启openldap:enabled: false #轻量级目录协议，开启。elkPrefix: logstashbasicAuth:enabled: false #这个不用改，系统默认的 logstash 没密码。username: ""password: ""auditing:               enabled: false #告警功能，要打开。auditing: #审计，查账enabled: false #这个要打开。     devops:                  # (CPU: 0.47 Core, Memory: 8.6 G) Provide an out-of-the-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image.enabled: false  #重点体验的功能，要打开。events:enabled: false #事件功能，打开。logging:enabled: false #日志功能，打开。metrics_server: #注意，这个已经安装了，别打开。这是从官方装的，经常下载不下来。enabled: falsenetwork:networkpolicy: enabled: false #网络策略，打开。ippool: type: none #Specify "calico" 。这个改为 calicoopenpitrix: store:enabled: false #应用商店，打开。servicemesh:       enabled: false #微服务治理功能，打开。kubeedge:          enabled: false  #边缘计算，因为没设备。打开吧，无所谓。体验不到。

进行安装

kubectl apply -f kubesphere-installer.yaml
kubectl apply -f cluster-configuration.yaml

#查看进度
kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

访问任意机器的 30880端口

账号： admin

密码： P@88w0rd

安装完毕信息打印

**************************************************
Collecting installation results ...
#####################################################
###              Welcome to KubeSphere!           ###
#####################################################Console: http://172.31.0.10:30880
Account: admin
Password: P@88w0rd。改为了：123456aANOTES：1. After you log into the console, please check themonitoring status of service components in"Cluster Management". If any service is notready, please wait patiently until all componentsare up and running.2. Please change the default password after login.#####################################################
https://kubesphere.io             2023-01-15 16:17:49
#####################################################

解决etcd监控证书找不到问题

kubectl describe pod -n kubesphere-monitoring-system   prometheus-k8s-
# 老师的是0名字 报，挂载失败。FailedMount，我的是 1
# kube-etcd-client-cert" not foundWarning  FailedMount       2m5s                   kubelet            Unable to attach or mount volumes: unmounted volumes=[secret-kube-etcd-client-certs], unattached volumes=[prometheus-k8s-db prometheus-k8s-rulefiles-0 secret-kube-etcd-client-certs prometheus-k8s-token-jxczz config config-out tls-assets]: timed out waiting for the condition

解决办法：

kubectl -n kubesphere-monitoring-system create secret generic kube-etcd-client-certs  --from-file=etcd-client-ca.crt=/etc/kubernetes/pki/etcd/ca.crt  --from-file=etcd-client.crt=/etc/kubernetes/pki/apiserver-etcd-client.crt  --from-file=etcd-client.key=/etc/kubernetes/pki/apiserver-etcd-client.key

点击右上角的 demo，可以在官方体验 kubesphere

https://kubesphere.io/zh/docs/v3.3/quick-start/minimal-kubesphere-on-k8s/

要安装的服务

Start installing monitoring #监控
Start installing multicluster #多集群
Start installing openpitrix #应用商店
Start installing network #网络 隔离
Start installing alerting #告警
Start installing auditing #审计
Start installing devops #一站式 部署运营。
Start installing events #事件
Start installing kubeedge #使能边缘计算的开放平台
Start installing logging #日志
Start installing servicemesh #服务治理

DevOps（Development和Operations的组合词operations
n.
运营；运作；业务操作（operation 的复数）mesh
n.
网，网状物；错综复杂的局面，陷阱，圈套；网眼，网孔；网状结构，）
v.
啮合；吻合，匹配，适合；（使）交缠在一起；将……转化成网格，将……生成网格；以网捕捉

kubesphere-installer.yaml

---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:name: clusterconfigurations.installer.kubesphere.io
spec:group: installer.kubesphere.ioversions:- name: v1alpha1served: truestorage: truescope: Namespacednames:plural: clusterconfigurationssingular: clusterconfigurationkind: ClusterConfigurationshortNames:- cc---
apiVersion: v1
kind: Namespace
metadata:name: kubesphere-system---
apiVersion: v1
kind: ServiceAccount
metadata:name: ks-installernamespace: kubesphere-system---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:name: ks-installer
rules:
- apiGroups:- ""resources:- '*'verbs:- '*'
- apiGroups:- appsresources:- '*'verbs:- '*'
- apiGroups:- extensionsresources:- '*'verbs:- '*'
- apiGroups:- batchresources:- '*'verbs:- '*'
- apiGroups:- rbac.authorization.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- apiregistration.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- apiextensions.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- tenant.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- certificates.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- devops.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- monitoring.coreos.comresources:- '*'verbs:- '*'
- apiGroups:- logging.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- jaegertracing.ioresources:- '*'verbs:- '*'
- apiGroups:- storage.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- admissionregistration.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- policyresources:- '*'verbs:- '*'
- apiGroups:- autoscalingresources:- '*'verbs:- '*'
- apiGroups:- networking.istio.ioresources:- '*'verbs:- '*'
- apiGroups:- config.istio.ioresources:- '*'verbs:- '*'
- apiGroups:- iam.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- notification.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- auditing.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- events.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- core.kubefed.ioresources:- '*'verbs:- '*'
- apiGroups:- installer.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- storage.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- security.istio.ioresources:- '*'verbs:- '*'
- apiGroups:- monitoring.kiali.ioresources:- '*'verbs:- '*'
- apiGroups:- kiali.ioresources:- '*'verbs:- '*'
- apiGroups:- networking.k8s.ioresources:- '*'verbs:- '*'
- apiGroups:- kubeedge.kubesphere.ioresources:- '*'verbs:- '*'
- apiGroups:- types.kubefed.ioresources:- '*'verbs:- '*'---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: ks-installer
subjects:
- kind: ServiceAccountname: ks-installernamespace: kubesphere-system
roleRef:kind: ClusterRolename: ks-installerapiGroup: rbac.authorization.k8s.io---
apiVersion: apps/v1
kind: Deployment
metadata:name: ks-installernamespace: kubesphere-systemlabels:app: ks-install
spec:replicas: 1selector:matchLabels:app: ks-installtemplate:metadata:labels:app: ks-installspec:serviceAccountName: ks-installercontainers:- name: installerimage: kubesphere/ks-installer:v3.1.1imagePullPolicy: "Always"resources:limits:cpu: "1"memory: 1Girequests:cpu: 20mmemory: 100MivolumeMounts:- mountPath: /etc/localtimename: host-timevolumes:- hostPath:path: /etc/localtimetype: ""name: host-time

cluster-configuration.yaml

---
apiVersion: installer.kubesphere.io/v1alpha1
kind: ClusterConfiguration
metadata:name: ks-installernamespace: kubesphere-systemlabels:version: v3.1.1
spec:persistence:storageClass: ""        # If there is no default StorageClass in your cluster, you need to specify an existing StorageClass here.authentication:jwtSecret: ""           # Keep the jwtSecret consistent with the Host Cluster. Retrieve the jwtSecret by executing "kubectl -n kubesphere-system get cm kubesphere-config -o yaml | grep -v "apiVersion" | grep jwtSecret" on the Host Cluster.local_registry: ""        # Add your private registry address if it is needed.etcd:monitoring: true       # Enable or disable etcd monitoring dashboard installation. You have to create a Secret for etcd before you enable it.endpointIps: 172.31.0.4  # etcd cluster EndpointIps. It can be a bunch of IPs here.port: 2379              # etcd port.tlsEnable: truecommon:redis:enabled: trueopenldap:enabled: trueminioVolumeSize: 20Gi # Minio PVC size.openldapVolumeSize: 2Gi   # openldap PVC size.redisVolumSize: 2Gi # Redis PVC size.monitoring:# type: external   # Whether to specify the external prometheus stack, and need to modify the endpoint at the next line.endpoint: http://prometheus-operated.kubesphere-monitoring-system.svc:9090 # Prometheus endpoint to get metrics data.es:   # Storage backend for logging, events and auditing.# elasticsearchMasterReplicas: 1   # The total number of master nodes. Even numbers are not allowed.# elasticsearchDataReplicas: 1     # The total number of data nodes.elasticsearchMasterVolumeSize: 4Gi   # The volume size of Elasticsearch master nodes.elasticsearchDataVolumeSize: 20Gi    # The volume size of Elasticsearch data nodes.logMaxAge: 7                     # Log retention time in built-in Elasticsearch. It is 7 days by default.elkPrefix: logstash              # The string making up index names. The index name will be formatted as ks-<elk_prefix>-log.basicAuth:enabled: falseusername: ""password: ""externalElasticsearchUrl: ""externalElasticsearchPort: ""console:enableMultiLogin: true  # Enable or disable simultaneous logins. It allows different users to log in with the same account at the same time.port: 30880alerting:                # (CPU: 0.1 Core, Memory: 100 MiB) It enables users to customize alerting policies to send messages to receivers in time with different time intervals and alerting levels to choose from.enabled: true         # Enable or disable the KubeSphere Alerting System.# thanosruler:#   replicas: 1#   resources: {}auditing:                # Provide a security-relevant chronological set of records，recording the sequence of activities happening on the platform, initiated by different tenants.enabled: true         # Enable or disable the KubeSphere Auditing Log System. devops:                  # (CPU: 0.47 Core, Memory: 8.6 G) Provide an out-of-the-box CI/CD system based on Jenkins, and automated workflow tools including Source-to-Image & Binary-to-Image.enabled: true             # Enable or disable the KubeSphere DevOps System.jenkinsMemoryLim: 2Gi      # Jenkins memory limit.jenkinsMemoryReq: 1500Mi   # Jenkins memory request.jenkinsVolumeSize: 8Gi     # Jenkins volume size.jenkinsJavaOpts_Xms: 512m  # The following three fields are JVM parameters.jenkinsJavaOpts_Xmx: 512mjenkinsJavaOpts_MaxRAM: 2gevents:                  # Provide a graphical web console for Kubernetes Events exporting, filtering and alerting in multi-tenant Kubernetes clusters.enabled: true         # Enable or disable the KubeSphere Events System.ruler:enabled: truereplicas: 2logging:                 # (CPU: 57 m, Memory: 2.76 G) Flexible logging functions are provided for log query, collection and management in a unified console. Additional log collectors can be added, such as Elasticsearch, Kafka and Fluentd.enabled: true         # Enable or disable the KubeSphere Logging System.logsidecar:enabled: truereplicas: 2metrics_server:                    # (CPU: 56 m, Memory: 44.35 MiB) It enables HPA (Horizontal Pod Autoscaler).enabled: false                   # Enable or disable metrics-server.monitoring:storageClass: ""                 # If there is an independent StorageClass you need for Prometheus, you can specify it here. The default StorageClass is used by default.# prometheusReplicas: 1          # Prometheus replicas are responsible for monitoring different segments of data source and providing high availability.prometheusMemoryRequest: 400Mi   # Prometheus request memory.prometheusVolumeSize: 20Gi       # Prometheus PVC size.# alertmanagerReplicas: 1          # AlertManager Replicas.multicluster:clusterRole: none  # host | member | none  # You can install a solo cluster, or specify it as the Host or Member Cluster.network:networkpolicy: # Network policies allow network isolation within the same cluster, which means firewalls can be set up between certain instances (Pods).# Make sure that the CNI network plugin used by the cluster supports NetworkPolicy. There are a number of CNI network plugins that support NetworkPolicy, including Calico, Cilium, Kube-router, Romana and Weave Net.enabled: true # Enable or disable network policies.ippool: # Use Pod IP Pools to manage the Pod network address space. Pods to be created can be assigned IP addresses from a Pod IP Pool.type: calico # Specify "calico" for this field if Calico is used as your CNI plugin. "none" means that Pod IP Pools are disabled.topology: # Use Service Topology to view Service-to-Service communication based on Weave Scope.type: none # Specify "weave-scope" for this field to enable Service Topology. "none" means that Service Topology is disabled.openpitrix: # An App Store that is accessible to all platform tenants. You can use it to manage apps across their entire lifecycle.store:enabled: true # Enable or disable the KubeSphere App Store.servicemesh:         # (0.3 Core, 300 MiB) Provide fine-grained traffic management, observability and tracing, and visualized traffic topology.enabled: true     # Base component (pilot). Enable or disable KubeSphere Service Mesh (Istio-based).kubeedge:          # Add edge nodes to your cluster and deploy workloads on edge nodes.enabled: true   # Enable or disable KubeEdge.cloudCore:nodeSelector: {"node-role.kubernetes.io/worker": ""}tolerations: []cloudhubPort: "10000"cloudhubQuicPort: "10001"cloudhubHttpsPort: "10002"cloudstreamPort: "10003"tunnelPort: "10004"cloudHub:advertiseAddress: # At least a public IP address or an IP address which can be accessed by edge nodes must be provided.- ""            # Note that once KubeEdge is enabled, CloudCore will malfunction if the address is not provided.nodeLimit: "100"service:cloudhubNodePort: "30000"cloudhubQuicNodePort: "30001"cloudhubHttpsNodePort: "30002"cloudstreamNodePort: "30003"tunnelNodePort: "30004"edgeWatcher:nodeSelector: {"node-role.kubernetes.io/worker": ""}tolerations: []edgeWatcherAgent:nodeSelector: {"node-role.kubernetes.io/worker": ""}tolerations: []

5、其他安装方式

https://kubesphere.io/zh/docs/v3.3/quick-start/all-in-one-on-linux/

Linux单节点部署KubeSphere

1、开通服务器

4c8g；centos7.9；防火墙放行 30000~32767；指定hostname

hostnamectl set-hostname node1

2、安装

1、准备KubeKey

export KKZONE=cn#官网最新的为：VERSION=v3.0.2
curl -sfL https://get-kk.kubesphere.io | VERSION=v1.1.1 sh -chmod +x kk

2、使用KubeKey引导安装集群

#下面命令,会提示缺少什么：如 conntrack
yum install -y conntrack./kk create cluster --with-kubernetes v1.20.4 --with-kubesphere v3.1.1 #最新的为3.3.1

3、安装后开启功能

https://kubesphere.io/zh/docs/v3.3/pluggable-components/devops/

在安装后启用 DevOps

以 admin 用户登录控制台，点击左上角的平台管理，选择集群管理。
点击定制资源定义，在搜索栏中输入 clusterconfiguration，点击搜索结果查看其详细页面。

Linux多节点部署KubeSphere

https://kubesphere.io/zh/docs/v3.3/installing-on-linux/introduction/multioverview/

点官网，文档中心，选一个版本，在选多节点安装。

1、准备三台服务器

4c8g （master）
8c16g * 2（worker）
centos7.9
内网互通
每个机器有自己域名
防火墙开放30000~32767端口

2、使用KubeKey创建集群

1、下载KubeKey

export KKZONE=cncurl -sfL https://get-kk.kubesphere.io | VERSION=v1.1.1 sh -chmod +x kk

2、创建集群配置文件

./kk create config --with-kubernetes v1.20.4 --with-kubesphere v3.1.1

3、创建集群

vim config-sample.yaml

# 每个集群都要改，包括主节点。
spec:hosts:- {name: node1主机名, address: 172.16.0.2内网地址, internalAddress: 172.16.0.2内网地址, user: ubuntu, password: Qcloud@123}roleGroups:etcd:- node1 #装载主节点master: - node1 #改为主节点worker:- master #master 也能当node- node1- node2

# 缺什么插件，就补什么插件。
yum install -y conntrack
./kk create cluster -f config-sample.yaml

连接跟踪（CONNTRACK），顾名思义，就是跟踪并且记录连接状态。Linux为每一个经过网络堆栈的数据包，生成一个新的连接记录项 （Connection entry）。

track
英
/træk
n.
小道，小径；踪迹，足迹，车辙；（铁路的）轨道；（窗帘、推拉门等的）滑竿，滑轨；（航海）
v.
追查，追踪；（尤指用特殊电子设备）跟踪，追踪；跟踪（进展情况）；跟踪摄影，移动摄影；（录制设备的移动部件）运转正常；

4、查看进度

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

其他知识看官网

高可用配置

持久化存储配置

在公共云上安装在本地环境中安装

添加或删除节点