【K8S实战】-超详细教程（三）

1、存储

1.1、nfs默认存储

我这里只演示nfs作为K8S的默认存储，其他的可以看这里【存储类】。

1.1.1、安装nfs服务

所有机器都安装nfs工具

所有机器
[root@master ~]# yum install -y nfs-utils以下都是在master节点操作
[root@master ~]# echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports
[root@master ~]# mkdir -p /nfs/data
设置开机启动且立即启动
[root@master ~]# systemctl enable rpcbind --now
[root@master ~]# systemctl enable nfs-server --now
使配置生效
[root@master ~]# exportfs -r
检查配置是否生效
[root@master ~]# exportfs

配置nfs-client（可以不做）

查看nfs服务的共享文件目录
[root@node1 ~]# showmount -e 172.168.200.130创建并挂载目录
[root@node1 ~]# mkdir -p /nfs/data
[root@node1 ~]# mount -t nfs 172.168.200.130:/nfs/data /nfs/data

1.1.2、配置默认存储

在master节点执行如下命令：

[root@master ~]# kubectl create -f default-storage.yaml

default-storage.yaml文件内容如下：

## 创建了一个存储类
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: nfs-storageannotations:storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:archiveOnDelete: "true"  ## 删除pv的时候，pv的内容是否要备份---
apiVersion: apps/v1
kind: Deployment
metadata:name: nfs-client-provisionerlabels:app: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
spec:replicas: 1strategy:type: Recreateselector:matchLabels:app: nfs-client-provisionertemplate:metadata:labels:app: nfs-client-provisionerspec:serviceAccountName: nfs-client-provisionercontainers:- name: nfs-client-provisionerimage: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2# resources:#    limits:#      cpu: 10m#    requests:#      cpu: 10mvolumeMounts:- name: nfs-client-rootmountPath: /persistentvolumesenv:- name: PROVISIONER_NAMEvalue: k8s-sigs.io/nfs-subdir-external-provisioner- name: NFS_SERVERvalue: master ## 指定自己nfs服务器地址- name: NFS_PATH  value: /nfs/data  ## nfs服务器共享的目录volumes:- name: nfs-client-rootnfs:server: master  ## 指定自己nfs服务器地址path: /nfs/data  ## nfs服务器共享的目录
---
apiVersion: v1
kind: ServiceAccount
metadata:name: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: nfs-client-provisioner-runner
rules:- apiGroups: [""]resources: ["nodes"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["persistentvolumes"]verbs: ["get", "list", "watch", "create", "delete"]- apiGroups: [""]resources: ["persistentvolumeclaims"]verbs: ["get", "list", "watch", "update"]- apiGroups: ["storage.k8s.io"]resources: ["storageclasses"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["events"]verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: run-nfs-client-provisioner
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: ClusterRolename: nfs-client-provisioner-runnerapiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
rules:- apiGroups: [""]resources: ["endpoints"]verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: leader-locking-nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
subjects:- kind: ServiceAccountname: nfs-client-provisioner# replace with namespace where provisioner is deployednamespace: default
roleRef:kind: Rolename: leader-locking-nfs-client-provisionerapiGroup: rbac.authorization.k8s.io

有可能k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2下载不了，使用这个【离线镜像】，提取码：u2hi，然后在所有机器执行如下命令：

[root@node1 ~]# docker load -i nfs-subdir-external-provisioner-v4.0.2.tar

1.2、PV与PVC

PV 卷是集群中的资源。PVC 申领是对这些资源的请求，也被用来执行对资源的申领检查。

1.2.1 pv

持久卷（PersistentVolume，PV）是集群中的一块存储，可以由管理员事先供应，或者使用存储类（Storage Class）来动态供应。
资源pv.yaml内容如下：

apiVersion: v1
kind: PersistentVolume
metadata:name: pv01-10m
spec:capacity:storage: 10MaccessModes:- ReadWriteManystorageClassName: nfsnfs:path: /nfs/data/01 #nfs共享的文件目录server: 172.168.200.130 #这里是nfs服务器
---
apiVersion: v1
kind: PersistentVolume
metadata:name: pv02-1gi
spec:capacity:storage: 1GiaccessModes:- ReadWriteManystorageClassName: nfsnfs:path: /nfs/data/02server: 172.168.200.130 #这里是nfs服务器
---
apiVersion: v1
kind: PersistentVolume
metadata:name: pv03-3gi
spec:capacity:storage: 3GiaccessModes:- ReadWriteManystorageClassName: nfsnfs:path: /nfs/data/03server: 172.168.200.130 #这里是nfs服务器

1.2.1.1、静态供应

在nfs服务上创建如下目录
[root@master pv-pvc]# mkdir -p /nfs/data/01 /nfs/data/02 /nfs/data/03
[root@master pv-pvc]# ll /nfs/data/
total 0
drwxr-xr-x. 2 root root 6 Jul  1 14:05 01
drwxr-xr-x. 2 root root 6 Jul  1 14:05 02
drwxr-xr-x. 2 root root 6 Jul  1 14:05 03创建pv
[root@master pv-pvc]# kubectl create -f pv.yaml
persistentvolume/pv01-10m created
persistentvolume/pv02-1gi created
persistentvolume/pv03-3gi created
查看刚刚创建的pv
[root@master pv-pvc]# kubectl get pv
NAME       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
pv01-10m   10M        RWX            Retain           Available           nfs                     11s
pv02-1gi   1Gi        RWX            Retain           Available           nfs                     11s
pv03-3gi   3Gi        RWX            Retain           Available           nfs                     10s

1.2.2、pvc

持久卷申领（PersistentVolumeClaim，PVC）表达的是用户对存储的请求。概念上与 Pod 类似。
资源pvc.yaml内容如下：

kind: PersistentVolumeClaim
apiVersion: v1
metadata:name: temp-pvc
spec:accessModes:- ReadWriteManyresources:requests:storage: 200MistorageClassName: nfs

1.2.2.1、pvc绑定pv

使用资源pvc.yaml对pv进行申请，发现会绑定到1G的pv卷上了，pvc规则就是杜绝浪费，目标pv>=pvc所申请的空间。

创建pvc并绑定pv
[root@master pv-pvc]# kubectl apply -f pvc.yaml
persistentvolumeclaim/temp-pvc created查看相关资源果然如上述所料那样，temp-pvc选择了1G的pv卷
[root@master pv-pvc]# kubectl get pv,pvc
NAME                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM              STORAGECLASS   REASON   AGE
persistentvolume/pv01-10m   10M        RWX            Retain           Available                      nfs                     4m40s
persistentvolume/pv02-1gi   1Gi        RWX            Retain           Bound       default/temp-pvc   nfs                     4m40s
persistentvolume/pv03-3gi   3Gi        RWX            Retain           Available                      nfs                     4m39sNAME                             STATUS   VOLUME     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/temp-pvc   Bound    pv02-1gi   1Gi        RWX            nfs            33s

1.2.2.2、pod申请pvc

资源nginx-pvc.yaml内容如下：

apiVersion: apps/v1
kind: Deployment
metadata:labels:app: nginx-test-pvcname: nginx-test-pvc
spec:replicas: 2selector:matchLabels:app: nginx-test-pvctemplate:metadata:labels:app: nginx-test-pvcspec:containers:- image: nginxname: nginxvolumeMounts:- name: htmlmountPath: /usr/share/nginx/htmlvolumes:- name: htmlpersistentVolumeClaim:claimName: temp-pvc #这里是我们上一步创建的pvc名称

实验演示开始：

目前环境运行的初始状态
[root@master ~]# kubectl get pod -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
default       nfs-client-provisioner-846cc9c4-4687x      1/1     Running   0          26h
kube-system   calico-kube-controllers-6d9cdcd744-tcjnc   1/1     Running   0          28h
kube-system   calico-node-pbmzt                          1/1     Running   0          28h
kube-system   calico-node-w7dd4                          1/1     Running   0          28h
kube-system   coredns-5897cd56c4-7fgpb                   1/1     Running   0          28h
kube-system   coredns-5897cd56c4-gzbk2                   1/1     Running   0          28h
kube-system   etcd-master                                1/1     Running   0          28h
kube-system   kube-apiserver-master                      1/1     Running   0          28h
kube-system   kube-controller-manager-master             1/1     Running   0          28h
kube-system   kube-proxy-9p7vg                           1/1     Running   0          28h
kube-system   kube-proxy-c5cdp                           1/1     Running   0          28h
kube-system   kube-scheduler-master                      1/1     Running   0          28h创建pod并申请pvc资源
[root@master pv-pvc]# kubectl apply -f nginx-pvc.yaml
deployment.apps/nginx-test-pvc created
稍等一会查看全部都是运行状态
[root@master pv-pvc]# kubectl get pod -A -owide
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE    IP                NODE     NOMINATED NODE   READINESS GATES
default       nfs-client-provisioner-846cc9c4-4687x      1/1     Running   0          26h    192.168.166.129   node1    <none>           <none>
default       nginx-test-pvc-54f98f8c6-4l9g7             1/1     Running   0          2m8s   192.168.166.163   node1    <none>           <none>
default       nginx-test-pvc-54f98f8c6-bt88x             1/1     Running   0          2m8s   192.168.166.162   node1    <none>           <none>
......
kube-system   kube-scheduler-master                      1/1     Running   0          28h    172.168.200.130   master   <none>           <none>访问nginx，报403。这是因为我们挂载的/nfs/data/02没有创建index.html。
[root@master pv-pvc]# curl 192.168.166.163
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.21.5</center>
</body>
</html>
[root@master pv-pvc]# 创建index.html，重新访问就ok了
[root@master pv-pvc]# echo ”欢迎来到德莱联盟“ >> /nfs/data/02/index.html
[root@master pv-pvc]# curl 192.168.166.163
”欢迎来到德莱联盟“
[root@master pv-pvc]# curl 192.168.166.162
”欢迎来到德莱联盟“

注：无状态服务多个副本都是使用同一个pvc。

1.3、ConfigMap（配置集）

ConfigMap 主要定义一些pod’需要的配置文件（偶尔变动）。例如环境变量、命令参数等等。
创建一个redis的配置集：

查看配置集创建命令
[root@master confMap]# kubectl create configmap --help
[root@master confMap]# kubectl create configmap my-redis-conf --from-file=redis.conf
error: error reading redis.conf: no such file or directory
[root@master confMap]# vi redis.conf
[root@master confMap]# kubectl create configmap my-redis-conf --from-file=redis.conf
configmap/my-redis-conf created
[root@master confMap]#

如需使用资源创建配置集，其yaml内容如下：

apiVersion: v1
kind: ConfigMap
metadata:name: my-redis-confnamespace: default
data:    #data是所有真正的数据，key：默认是文件名   value：配置文件的内容redis.conf: |appendonly yes

redis绑定my-redis-conf配置集，redis.yaml如下：

apiVersion: v1
kind: Pod
metadata:name: redis
spec:containers:- name: redisimage: rediscommand:- redis-server #命令，后续的都是参数- "/redis-master/redis.conf"  #指的是redis容器内部的位置ports:- containerPort: 6379volumeMounts:- mountPath: /dataname: data #挂载的名称，对应volumes下的配置- mountPath: /redis-master #容器内的路径name: config #挂载的名称，对应volumes下的配置volumes:- name: dataemptyDir: {}- name: configconfigMap:name: my-redis-conf #我们创建配置集的nameitems:- key: redis.conf #配置文件名，这里等于创建配置集时的--from-file=redis.confpath: redis.conf

[root@master confMap]# kubectl apply -f redis.yaml
pod/redis created[root@master confMap]# kubectl get pod -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
default       nfs-client-provisioner-846cc9c4-4687x      1/1     Running   0          23h
default       redis                                      1/1     Running   0          32s
......
kube-system   kube-scheduler-master                      1/1     Running   0          25h查看redis的appedonly默认配置
[root@master confMap]# kubectl exec -it redis -- redis-cli config get appendonly
1) "appendonly"
2) "no"修改config配置集
[root@master confMap]# kubectl edit cm my-redis-conf
configmap/my-redis-conf editeddata:redis.conf: ”“
改成下面这种，然后保存即可
data:redis.conf: |appendonly yes重启redis，因为配置修改后redis无法感知配置发生变化。
[root@master confMap]# kubectl delete -f redis.yaml
[root@master confMap]# kubectl apply-f redis.yaml查看更新后的配置
[root@master confMap]# kubectl exec -it redis -- redis-cli config get appendonly
1) "appendonly"
2) "yes"

1.4、Secret

Secret 是一种包含少量敏感信息例如密码、令牌或密钥的对象。这样的信息可能会被放在 Pod 规约中或者镜像中。使用 Secret 意味着你不需要在应用程序代码中包含机密数据。说白了就是保护敏感数据。

1.3.1、命令方式创建

kubectl create secret docker-registry regcred \--docker-server=<你的镜像仓库服务器> \--docker-username=<你的用户名> \--docker-password=<你的密码> \--docker-email=<你的邮箱地址>

1.3.2、Yaml方式创建

Secret 的类型有很多种，这里使用了Opaque。

apiVersion: v1
kind: Secret
metadata:name: mysecret
type: Opaque
data:USER_NAME: YWRtaW4= #Base64 字符串PASSWORD: MWYyZDFlMmU2N2Rm #Base64 字符串

[root@master confMap]# kubectl apply -f secret.yaml
secret/mysecret created

1.3.3、使用Secret

redis-secret.yaml文件内容如下：

apiVersion: v1
kind: Pod
metadata:name: mypod
spec:containers:- name: mypodimage: redisvolumeMounts:- name: foomountPath: "/etc/foo"readOnly: truevolumes:- name: foosecret:secretName: mysecretoptional: false # 默认设置，意味着 "mysecret" 必须已经存在

先把之前的停了，便于观察
[root@master confMap]# kubectl delete -f redis.yaml
pod "redis" deleted
[root@master confMap]# kubectl get secret
NAME                                 TYPE                                  DATA   AGE
default-token-zgjgg                  kubernetes.io/service-account-token   3      25h
mysecret                             Opaque                                2      18s
nfs-client-provisioner-token-426rh   kubernetes.io/service-account-token   3      24h[root@master confMap]# kubectl apply -f redis-secret.yaml
pod/mypod created
[root@master confMap]# kubectl get pod -A
NAMESPACE     NAME                                       READY   STATUS    RESTARTS   AGE
default       mypod                                      1/1     Running   0          5m21s
......
kube-system   kube-scheduler-master                      1/1     Running   0          27h
[root@master confMap]# 查看变量配置，发现密码已经被解码了
[root@master confMap]#  kubectl exec -it mypod --  ls /etc/foo
PASSWORD  USER_NAME
[root@master confMap]#  kubectl exec -it mypod --  cat /etc/foo/USER_NAME
admin
[root@master confMap]#  kubectl exec -it mypod --  cat /etc/foo/PASSWORD
1f2d1e2e67df

环境篇：k8s搭建（超详细，保姆级教程）
上一篇：【K8S实战】-超详细教程（二）
下一篇：【K8S实战】-超详细教程（四）——孵化中

资料参考：

K8S官网