K8S使用NFS动态存储（csi-nfs-driver）
2024-05-10 00:34:23
K8S使用NFS动态存储（csi-nfs-driver）
- 配置nfs服务器
- k8s-master部署csi-nfs-driver插件
- 测试创建pod，使用local-storage动态创建pvc
- nfs服务器查看挂载
配置nfs服务器
[root@localhost ~]#yum install nfs-utils rpcbind -y
[root@localhost ~]#mkdir -p /data/nfs/
[root@localhost ~]#chmod 755 /data/nfs/
[root@localhost ~]#vim /etc/exports
/data/nfs/ *(insecure,rw,sync,no_root_squash,no_all_squash) //nfs挂载目录；*：挂载地址（*代表所有）
[root@localhost ~]#systemctl start nfs
[root@localhost ~]#systemctl start rpcbind
k8s-master部署csi-nfs-driver插件
rbac-csi-nfs-controller.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:name: csi-nfs-controller-sanamespace: kube-system---kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: nfs-external-provisioner-role
rules:- apiGroups: [""]resources: ["persistentvolumes"]verbs: ["get", "list", "watch", "create", "delete"]- apiGroups: [""]resources: ["persistentvolumeclaims"]verbs: ["get", "list", "watch", "update"]- apiGroups: ["storage.k8s.io"]resources: ["storageclasses"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["events"]verbs: ["get", "list", "watch", "create", "update", "patch"]- apiGroups: ["storage.k8s.io"]resources: ["csinodes"]verbs: ["get", "list", "watch"]- apiGroups: [""]resources: ["nodes"]verbs: ["get", "list", "watch"]- apiGroups: ["coordination.k8s.io"]resources: ["leases"]verbs: ["get", "list", "watch", "create", "update", "patch"]
---kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:name: nfs-csi-provisioner-binding
subjects:- kind: ServiceAccountname: csi-nfs-controller-sanamespace: kube-system
roleRef:kind: ClusterRolename: nfs-external-provisioner-roleapiGroup: rbac.authorization.k8s.io
csi-nfs-driverinfo.yaml:
---
apiVersion: storage.k8s.io/v1beta1
kind: CSIDriver
metadata:name: nfs.csi.k8s.io
spec:attachRequired: falsevolumeLifecycleModes:- PersistentpodInfoOnMount: true
csi-nfs-node.yaml：
---
# This YAML file contains driver-registrar & csi driver nodeplugin API objects
# that are necessary to run CSI nodeplugin for nfs
kind: DaemonSet
apiVersion: apps/v1
metadata:name: csi-nfs-nodenamespace: kube-system
spec:selector:matchLabels:app: csi-nfs-nodetemplate:metadata:labels:app: csi-nfs-nodespec:hostNetwork: true # original nfs connection would be broken without hostNetwork settingdnsPolicy: ClusterFirstWithHostNetcontainers:- name: liveness-probeimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/livenessprobe:v2.1.0args:- --csi-address=/csi/csi.sock- --probe-timeout=3s- --health-port=29653- --v=5volumeMounts:- name: socket-dirmountPath: /csiresources:limits:cpu: 100mmemory: 100Mirequests:cpu: 10mmemory: 20Mi- name: node-driver-registrarimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/csi-node-driver-registrar:v2.0.1lifecycle:preStop:exec:command: ["/bin/sh", "-c", "rm -rf /registration/csi-nfsplugin /registration/csi-nfsplugin-reg.sock"]args:- --v=5- --csi-address=/csi/csi.sock- --kubelet-registration-path=/var/lib/kubelet/plugins/csi-nfsplugin/csi.sockenv:- name: KUBE_NODE_NAMEvalueFrom:fieldRef:fieldPath: spec.nodeNamevolumeMounts:- name: socket-dirmountPath: /csi- name: registration-dirmountPath: /registration- name: nfssecurityContext:privileged: truecapabilities:add: ["SYS_ADMIN"]allowPrivilegeEscalation: trueimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/nfsplugin:amd64-linux-canaryargs:- "-v=5"- "--nodeid=$(NODE_ID)"- "--endpoint=$(CSI_ENDPOINT)"env:- name: NODE_IDvalueFrom:fieldRef:fieldPath: spec.nodeName- name: CSI_ENDPOINTvalue: unix:///csi/csi.sockimagePullPolicy: "IfNotPresent"volumeMounts:- name: socket-dirmountPath: /csi- name: pods-mount-dirmountPath: /var/lib/kubelet/podsmountPropagation: "Bidirectional"volumes:- name: socket-dirhostPath:path: /var/lib/kubelet/plugins/csi-nfsplugintype: DirectoryOrCreate- name: pods-mount-dirhostPath:path: /var/lib/kubelet/podstype: Directory- hostPath:path: /var/lib/kubelet/plugins_registrytype: Directoryname: registration-dir
csi-nfs-controller.yaml：
---
kind: Deployment
apiVersion: apps/v1
metadata:name: csi-nfs-controllernamespace: kube-system
spec:replicas: 2selector:matchLabels:app: csi-nfs-controllertemplate:metadata:labels:app: csi-nfs-controllerspec:serviceAccountName: csi-nfs-controller-sanodeSelector:kubernetes.io/os: linuxpriorityClassName: system-cluster-criticaltolerations:- key: "node-role.kubernetes.io/master"operator: "Equal"value: "true"effect: "NoSchedule"containers:- name: csi-provisionerimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/csi-provisioner:v2.0.4args:- "-v=5"- "--csi-address=$(ADDRESS)"- "--leader-election"env:- name: ADDRESSvalue: /csi/csi.sockvolumeMounts:- mountPath: /csiname: socket-dirresources:limits:cpu: 100mmemory: 100Mirequests:cpu: 10mmemory: 20Mi- name: liveness-probeimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/livenessprobe:v2.1.0args:- --csi-address=/csi/csi.sock- --probe-timeout=3s- --health-port=29652- --v=5volumeMounts:- name: socket-dirmountPath: /csiresources:limits:cpu: 100mmemory: 100Mirequests:cpu: 10mmemory: 20Mi- name: nfsimage: registry.cn-hangzhou.aliyuncs.com/imagesfromgoogle/nfsplugin:amd64-linux-canarysecurityContext:privileged: truecapabilities:add: ["SYS_ADMIN"]allowPrivilegeEscalation: trueimagePullPolicy: IfNotPresentargs:- "-v=5"- "--nodeid=$(NODE_ID)"- "--endpoint=$(CSI_ENDPOINT)"env:- name: NODE_IDvalueFrom:fieldRef:fieldPath: spec.nodeName- name: CSI_ENDPOINTvalue: unix:///csi/csi.sockvolumeMounts:- name: pods-mount-dirmountPath: /var/lib/kubelet/podsmountPropagation: "Bidirectional"- mountPath: /csiname: socket-dirresources:limits:cpu: 200mmemory: 200Mirequests:cpu: 10mmemory: 20Mivolumes:- name: pods-mount-dirhostPath:path: /var/lib/kubelet/podstype: Directory- name: socket-diremptyDir: {}
storageclass-nfs.yaml ,修改以下的nfs的地址和目录
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:name: local-storage
provisioner: nfs.csi.k8s.io
parameters:server: 20.0.0.21share: /data/nfs/
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
mountOptions:- vers=3- noresvport
[root@k8s-master1 ~]#kubectl apply -f rbac-csi-nfs-controller.yaml
[root@k8s-master1 ~]#kubectl apply -f csi-nfs-driverinfo.yaml
[root@k8s-master1 ~]#kubectl apply -f csi-nfs-node.yaml
[root@k8s-master1 ~]#kubectl apply -f csi-nfs-controller.yaml
[root@k8s-master1 ~]#kubectl apply -f storageclass-nfs.yaml[root@k8s-master1 ~]#kubectl get po -n kube-system|grep csi-nfs
csi-nfs-controller-6745649bd6-jz9lp 3/3 Running 0 53m
csi-nfs-controller-6745649bd6-rtw95 3/3 Running 0 53m
csi-nfs-node-7krpw 3/3 Running 0 53m
csi-nfs-node-j2pfp 3/3 Running 0 53m
csi-nfs-node-smp9s 3/3 Running 0 53m
测试创建pod，使用local-storage动态创建pvc
apiVersion: apps/v1
kind: StatefulSet
metadata:name: web
spec:serviceName: "nginx"replicas: 2selector:matchLabels:app: nginxtemplate:metadata:labels:app: nginxspec:containers:- name: nginximage: nginxports:- containerPort: 80name: webvolumeMounts:- name: wwwmountPath: /usr/share/nginx/htmlvolumeClaimTemplates:- metadata:name: wwwspec:accessModes: [ "ReadWriteOnce" ]resources:requests:storage: 5GistorageClassName: local-storage
[root@k8s-master1 ~]#kubectl get pv
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
pvc-a6b65499-8acf-4c81-9cae-2df3acbeadec 5Gi RWO Delete Bound default/www-web-1 local-storage 10m
pvc-c2ce6bec-0f16-4984-9e07-fb6fa876abbf 5Gi RWO Delete Bound default/www-web-0 local-storage 10m
[root@k8s-master1 ~]#kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
www-web-0 Bound pvc-c2ce6bec-0f16-4984-9e07-fb6fa876abbf 5Gi RWO local-storage 10m
www-web-1 Bound pvc-a6b65499-8acf-4c81-9cae-2df3acbeadec 5Gi RWO local-storage 10m[root@k8s-master1 ~]#kubectl get pod
NAME READY STATUS RESTARTS AGE
web-0 1/1 Running 0 43s
web-1 1/1 Running 0 25s
nfs服务器查看挂载
[root@localhost ~]#cd /data/nfs/
[root@localhost nfs]#ll
总用量 0
drwxr-xr-x. 2 root root 6 12月 9 16:40 pvc-a6b65499-8acf-4c81-9cae-2df3acbeadec
drwxr-xr-x. 2 root root 6 12月 9 16:39 pvc-c2ce6bec-0f16-4984-9e07-fb6fa876abbf
K8S使用NFS动态存储（csi-nfs-driver）相关推荐
- K8S使用持久化卷存储到NFS（NAS盘）
参考文章:K8S-v1.20中使用PVC持久卷 - 知乎 目录 1.概念: 1.1 基础概念 1.2 PV的配置 1.2.1 静态PV配置 1.2.2 动态PV配置 1.2.3 PVC与PV的绑定 1 ...
- centos 7.6 ——NFS共享存储服务
NFS共享存储服务 文章目录 NFS共享存储服务 nfs 原理 一.实验准备 三.实验步骤 服务端192.168.75.137配置 客户机192.168.75.134配置 查看服务端文件 注意 FTP ...
- Linux网络系列--YUM仓库部署与NFS服务（YUM本地源仓库与远程ftp仓库部署、YUM命令使用、NFS共享存储服务讲解及实验）
文章目录 一. 构建YUM软件仓库 1.1 本地源YUM仓库配置 1.2 远程yum仓库(借助FTP协议) 1.2.1 准备网络安装源(服务器端) 1.2.2 配置软件仓库位置(客户机端) 二. YU ...
- Docker基于容器制作httpd镜像(使用nfs共享存储部署一个网站)
Docker基于容器制作httpd镜像 1 编译安装httpd 2 安装nfs共享存储 3 使用nfs共享存储提供httpd网页服务 4 在浏览器上进行测试访问 1 编译安装httpd [root@d ...
- Docker(二十九)k8s 创建动态存储，基于nfs 的storageclass
可参照官方的配置 https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploy 第一种 ...
- K8S持久化存储：NFS+StorageClass实现动态供给
K8S持久化存储:NFS+StorageClass实现动态供给 一.简介:K8S中的持久化存储 二.部署nfs-provisioner 三.创建StorageClass 四.实验 实验1:部署depl ...
- 使用nfs为k8s提供pv动态供给存储
环境:k8s一主两从 k8s version: 1.20.2 k8s-master: 192.168.31.200 k8s-node1: 192.168.31.201 k8s-node2: 192.1 ...
- k8s—centos7安装部署NFS服务器和客户端及基于nfs的动态存储storageclass使用总结
并通过StatefulSet动态为pod生成pv挂载 简介 1. 安装配置前置环境 1.1 安装nfs文件系统 1.1. ...
最新文章
- jms在jboss上的简单应用
- exchange 只发送邮件 不能登录_springboot + rabbitmq发送邮件
- Happy Necklace dp 递推 矩阵快速幂
- 数据科学中的Docker
- Java使用ojdbc连接Oracle数据库时不能使用服务名连接的问题
- .net持续集成sonarqube篇之sonarqube安装与基本配置
- 无服务器冷启动不是问题-这就是为什么（对于大多数应用程序）
- Spring事务配置的五种方式和spring里面事务的传播属性和事务隔离级别、不可重复读与幻读的区别
- SQL Server中常用全局变量介绍
- pku773_Happy 2006
- (How to)Windows Live Writer插入Latex公式（补充）
- 值得收藏的前端大牛博客
- hdoj--5621--KK's Point（简单数学）
- [书目20080630]人一生要养成的50个习惯
- mysql 5.6忘记root密码_Linux平台MySQL忘记root密码解决方案
- MATLAB+三对角矩阵
- Unity创建注册表并写入
- Crazy bubbles
- R语言入门——一文讲明白attach与detach
- 一个很不错的C++类的练习题目