Helm安装Harbor
2024-05-01 11:19:35
前言
文章中用到的harbor存储是hostpath，service的类型是ingress。根据harbor官网提示，需要提前创建pv和pvc，既然选择了hostpath，那么就需要将pod固定到某一个node上面，本文所有的资源副本都是1，harbor的所有pod放在同一个namespace，并且需要将namespace固定到某个node上，具体步骤见正文。
一.环境准备
1.二进制安装helm
[root@k8s-master helm]# wget https://get.helm.sh/helm-v3.10.3-linux-amd64.tar.gz
[root@k8s-master helm]# tar zxf helm-v3.10.3-linux-amd64.tar.gz
[root@k8s-master helm]# cd linux-amd64/
[root@k8s-master helm]# cp helm /usr/local/bin/
[root@k8s-master helm]# helm version
version.BuildInfo{Version:"v3.10.3", GitCommit:"835b7334cfe2e5e27870ab3ed4135f136eecc704", GitTreeState:"clean", GoVersion:"go1.18.9"}
2.创建namespace
[root@k8s-master helm]# cat namespace-harbor.yaml
apiVersion: v1
kind: Namespace
metadata:name: harbor
[root@k8s-master helm]# kubectl apply -f namespace-harbor.yaml
3.开启准入控制器
–enable-admission-plugins 添加 PodNodeSelector
[root@k8s-master helm]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
apiVersion: v1
kind: Pod
metadata:annotations:kubeadm.kubernetes.io/kube-apiserver.advertise-address.endpoint: 192.168.0.20:6443creationTimestamp: nulllabels:component: kube-apiservertier: control-planename: kube-apiservernamespace: kube-system
spec:containers:- command:- kube-apiserver- --advertise-address=192.168.0.20- --allow-privileged=true- --authorization-mode=Node,RBAC- --client-ca-file=/etc/kubernetes/pki/ca.crt- --enable-admission-plugins=NodeRestriction,PodNodeSelector
apiserver pod会自动重启
4.命名空间加注解
[root@k8s-master helm]# kubectl edit ns harbor -o yaml
apiVersion: v1
kind: Namespace
metadata:annotations:scheduler.alpha.kubernetes.io/node-selector: harbor=envname: harbor
5.node打标签
[root@k8s-master helm]# kubectl label node k8s-node3 harbor=env
6.创建pv和pvc
选择hostpath，需要提前准备pv和pvc，官网解释如下：
官网链接：https://goharbor.io/docs/2.7.0/install-config/harbor-ha-helm/
在k8s-node3节点创建目录，建议给/data下单独挂一块存储盘
[root@k8s-node3 data]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/vdb 500G 98M 500G 1% /data
[root@k8s-node3 data]# mkdir -pv harbor/{chartmuseum,database,jobservice,redis,registry,scandata,trivy}
mkdir: created directory ‘harbor’
mkdir: created directory ‘harbor/chartmuseum’
mkdir: created directory ‘harbor/database’
mkdir: created directory ‘harbor/jobservice’
mkdir: created directory ‘harbor/redis’
mkdir: created directory ‘harbor/registry’
mkdir: created directory ‘harbor/scandata’
mkdir: created directory ‘harbor/trivy’
[root@k8s-node3 data]# chmod 777 -R harbor/ ##必须要赋权，不然pod启动会有问题
创建pv和pvc
[root@k8s-master helm]# cat harbor-pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-registry-pv"labels:name: harbor-registry-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/registrytype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-chartmuseum-pv"labels:name: harbor-chartmuseum-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/chartmuseumtype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-jobservice-pv"labels:name: harbor-jobservice-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/jobservicetype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-database-pv"labels:name: harbor-database-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/databasetype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-redis-pv"labels:name: harbor-redis-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/redistype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-trivy-pv"labels:name: harbor-trivy-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/trivytype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolume
metadata:name: "harbor-scandata-pv"labels:name: harbor-scandata-pvrelease: stable
spec:capacity:storage: 5GiaccessModes:- ReadWriteOncepersistentVolumeReclaimPolicy: RetainhostPath:path: /data/harbor/scandatatype: DirectoryOrCreate
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-registry-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-registry-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-chartmuseum-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-chartmuseum-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-jobservice-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-jobservice-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-database-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-database-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-redis-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-redis-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-trivy-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-trivy-pvrelease: stable
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:name: harbor-scandata-pvcnamespace: harbor
spec:accessModes:- ReadWriteOnceresources:requests:storage: 5Giselector:matchLabels:name: harbor-scandata-pvrelease: stable
[root@k8s-master helm]# kubectl apply -f harbor-pv.yaml
[root@k8s-master helm]# kubectl get pv,pvc -n harbor
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE
persistentvolume/harbor-chartmuseum-pv 5Gi RWO Retain Bound harbor/harbor-chartmuseum-pvc 31h
persistentvolume/harbor-database-pv 5Gi RWO Retain Bound harbor/harbor-database-pvc 31h
persistentvolume/harbor-jobservice-pv 5Gi RWO Retain Bound harbor/harbor-jobservice-pvc 31h
persistentvolume/harbor-redis-pv 5Gi RWO Retain Bound harbor/harbor-redis-pvc 31h
persistentvolume/harbor-registry-pv 5Gi RWO Retain Bound harbor/harbor-registry-pvc 31h
persistentvolume/harbor-scandata-pv 5Gi RWO Retain Bound harbor/harbor-scandata-pvc 30h
persistentvolume/harbor-trivy-pv 5Gi RWO Retain Bound harbor/harbor-trivy-pvc 31hNAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/harbor-chartmuseum-pvc Bound harbor-chartmuseum-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-database-pvc Bound harbor-database-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-jobservice-pvc Bound harbor-jobservice-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-redis-pvc Bound harbor-redis-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-registry-pvc Bound harbor-registry-pv 5Gi RWO 31h
persistentvolumeclaim/harbor-scandata-pvc Bound harbor-scandata-pv 5Gi RWO 30h
persistentvolumeclaim/harbor-trivy-pvc Bound harbor-trivy-pv 5Gi RWO 31h二.安装harbor
1.添加harbor的helm源
[root@k8s-master helm]# helm repo add harbor https://helm.goharbor.io
[root@k8s-master helm]# helm search repo
NAME CHART VERSION APP VERSION DESCRIPTION
harbor/harbor 1.11.0 2.7.0 An open source trusted cloud native registry th...
[root@k8s-master helm]# helm pull harbor/harbor
[root@k8s-master helm]# tar zxf harbor-1.11.0.tgz
[root@k8s-master opt]# cd harbor/
[root@k8s-master harbor]# ls
cert Chart.yaml conf LICENSE README.md templates values.yaml
2.前方高能，修改values.yaml
这里用默认类型 ingress，https协议，如果想用http协议，那么需要删除掉“expose.ingress.annotations”里面的“ssl-redirect”相关注解
[root@k8s-master harbor]# vim values.yaml
expose:# Set how to expose the service. Set the type as "ingress", "clusterIP", "nodePort" or "loadBalancer"# and fill the information in the corresponding sectiontype: ingresstls:# Enable TLS or not.# Delete the "ssl-redirect" annotations in "expose.ingress.annotations" when TLS is disabled and "expose.type" is "ingress"# Note: if the "expose.type" is "ingress" and TLS is disabled,# the port must be included in the command when pulling/pushing images.# Refer to https://github.com/goharbor/harbor/issues/5291 for details.enabled: true # The source of the tls certificate. Set as "auto", "secret"# or "none" and fill the information in the corresponding section# 1) auto: generate the tls certificate automatically# 2) secret: read the tls certificate from the specified secret.# The tls certificate can be generated manually or by cert manager# 3) none: configure no tls certificate for the ingress. If the default# tls certificate is configured in the ingress controller, choose this optioncertSource: auto
修改持久化配置，将pvc name添加在existingClaim后面
persistence:enabled: true# Setting it to "keep" to avoid removing PVCs during a helm delete# operation. Leaving it empty will delete PVCs after the chart deleted# (this does not apply for PVCs that are created for internal database# and redis components, i.e. they are never deleted automatically)resourcePolicy: "keep"persistentVolumeClaim:registry:# Use the existing PVC which must be created manually before bound,# and specify the "subPath" if the PVC is shared with other componentsexistingClaim: "harbor-registry-pvc"# Specify the "storageClass" used to provision the volume. Or the default# StorageClass will be used (the default).# Set it to "-" to disable dynamic provisioningstorageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}chartmuseum:existingClaim: "harbor-chartmuseum-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}jobservice:jobLog:existingClaim: "harbor-jobservice-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}scanDataExports:existingClaim: "harbor-scandata-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}# If external database is used, the following settings for database will# be ignoreddatabase:existingClaim: "harbor-database-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}# If external Redis is used, the following settings for Redis will# be ignoredredis:existingClaim: "harbor-redis-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}trivy:existingClaim: "harbor-trivy-pvc"storageClass: ""subPath: ""accessMode: ReadWriteOncesize: 5Giannotations: {}
3.helm安装harbor
[root@k8s-master harbor]# helm install harbor . -f values.yaml -n harbor
Have a cup of coffee
[root@k8s-master harbor]# kubectl get pod -n harbor -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
harbor-chartmuseum-7b74f8b585-qcp89 1/1 Running 2 4h39m 10.244.107.231 k8s-node3 <none> <none>
harbor-core-7fc48678c8-qcdqk 1/1 Running 2 4h39m 10.244.107.233 k8s-node3 <none> <none>
harbor-database-0 1/1 Running 2 4h39m 10.244.107.236 k8s-node3 <none> <none>
harbor-jobservice-8486bb4bcb-2gjtp 1/1 Running 9 4h39m 10.244.107.237 k8s-node3 <none> <none>
harbor-notary-server-7f7bf8f6d-zphpg 1/1 Running 2 4h39m 10.244.107.244 k8s-node3 <none> <none>
harbor-notary-signer-5f9df848b7-skpxj 1/1 Running 2 4h39m 10.244.107.248 k8s-node3 <none> <none>
harbor-portal-748c6db9c6-qw97j 1/1 Running 2 4h39m 10.244.107.243 k8s-node3 <none> <none>
harbor-redis-0 1/1 Running 2 4h39m 10.244.107.228 k8s-node3 <none> <none>
harbor-registry-6777c99d8d-cb9rj 2/2 Running 4 4h39m 10.244.107.235 k8s-node3 <none> <none>
harbor-trivy-0 1/1 Running 2 4h39m 10.244.107.245 k8s-node3 <none> <none>
4.ingress绑定ingress class
[root@k8s-master harbor]# kubectl get ingress -n harbor
NAME CLASS HOSTS ADDRESS PORTS AGE
harbor-ingress <none> core.harbor.domain 192.168.0.21,192.168.0.22 80, 443 4h55m
[root@k8s-master harbor]# kubectl edit ingress harbor-ingress -n harbor
......
spec:ingressClassName: nginx ## k8s-1.20.X添加ingressClassNamerules:- host: core.harbor.domainhttp:paths:
......
[root@k8s-master harbor]# kubectl get ingress -n harbor ##查看添加成功
NAME CLASS HOSTS ADDRESS PORTS AGE
harbor-ingress nginx core.harbor.domain 192.168.0.21,192.168.0.22 80, 443 4h57m
5.服务器配置证书，不然docker login和docker pull 都会报错
在harbor-ingress 的secret里面找到data下的ca.crt
[root@k8s-master harbor]# kubectl get secret harbor-ingress -n harbor -o yaml
apiVersion: v1
data:ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRY3d4c0NtVmNPV2RpZWJTSVREcU9kakFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05Nakl4TWpJNE1EWXdOVE13V2hjTk1qTXhNakk0TURZdwpOVE13V2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRQ3g0YVBFbkJ5K2xicElsSytyZmlsdndIWkJRMzM1a3lxMThKWHhpcDl1SFQ0VkExUzEKbG45anVtQUR4Smk4QnhWZStmVlBDWW5UeGJoNWJua3pqdm9XekptOTlMRUY5UDJPdW5VZnZCWGt6VFZEK0o1TgpBZjhyMmtqMFRTck5yZVVKL3JwR01mT1V1MFRSSjg2N2g1WHg1aEpBeDNlWlk4LzRFWmZQOVFGRlFsVGRxRDFBCmlkUkRydjFCSGJIajNxVjU3ckI4L3VxMVJ1a2hjQUZodlZXKzMvdmNkelBaS0hjc0c4d3VhNml2ekExK096a1gKTFA2K2wvcnlPektWWG9kU3ZHV1RmVzJpSlFEUjJqSWliR1loSUF3SDIrZlU4MVJrdGNsS050b2hZYW5aYjdOdQpTcTlzOFB1b3Y2V1JDMWpwRUd6cjhtR2hKTEhtcHJMNlRXeDdBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVZc2xqdmpDZ3F5R2NRZ1ZWS0s5TnIreU8yWll3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFKd1NkMkFuMmpEdHp6NUIvOEl2UzliamxVN3hLVlR0M1pWWkc0N0R5aFJXTmRpVEs1NFF3dFdhCmtCYUZrV2xTQWJaUk5iOWwyak85NEFMelRCb0ttVU04QTBlZFBQbXdvTFFaZlV2NVp6SCtGM05PenNVT1I5dFIKbnhva3lkSU14WUlpQWp1YThudXRLK0g2TklsRDRiVVA1MVZSNFJsNjBhdHhSWi81OW1HMWw1ZUNVSWtObVJyaQpwY2FQTjBBa3NzR2dCNmpxK3VXMXhPU0l1M21idVl3amdzTEpqWTFkNkNuS1IvQ1RRZnRBKzB2NFNVRTJBTFRGClg0TEFRVXFsM0F2NU8wWkxFQXAzNE5aR0xGNCtnRmJuYi84MmdYTlBuY2IzQkxOR0tKVE5zL0RGTTBkYlgwajcKY3VIMHQvT01oV2YyZ084eTNBMDB6dkIzcTVZSkIwdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURVekNDQWp1Z0F3SUJBZ0lSQU9QeW44V1FlOVZOaUlYL21LSGhrSHN3RFFZSktvWklodmNOQVFFTEJRQXcKRkRFU01CQUdBMVVFQXhNSmFHRnlZbTl5TFdOaE1CNFhEVEl5TVRJeU9EQTJNRFV6TUZvWERUSXpNVEl5T0RBMgpNRFV6TUZvd0hURWJNQmtHQTFVRUF4TVNZMjl5WlM1b1lYSmliM0l1Wkc5dFlXbHVNSUlCSWpBTkJna3Foa2lHCjl3MEJBUUVGQUFPQ0FROEFNSUlCQ2dLQ0FRRUF2dno0N0l4d2JUZDBQOFcvWngxUTVYaS9IRWZLajJmRzFlK08KQ0RMb2psSWEvbmErUlpaRzZvSVNIbGt6TVo2cTh3WmhZeTRVRTFCQi9VM1hKK2xuNHNhTkgrVllTQjdDbmFuYQpiWW5DVkp2Y091d1J3LzVaNUJzU1UreE55U0tLd3dGN0hLcDZBdklIc1puaU85aHhMNUVKRjRmbmtHY3BQZlZKCjNQTzdMY3B4ZDBVcEJ0enJyYnBWWTV1eDlmY3ZKSWNFeEJTY3ZmaGxmN3VjR0pYSDBZa0xra3RJUVAveWdRekMKQWMvUG82NDJ6VUJqNE1RWGVaRVQ2WjRxaWtLWnc4VXRGTyttZmZBU3BPT3FmMkpncDJiaU4yR1VqR044MTkxTAo1NU1BUUY3aFBzUWo0Y0k2aEVWczk3VmtXci9ZKzdxN3EyNkZnQ1IxY0p6ZVBQY1puUUlEQVFBQm80R1dNSUdUCk1BNEdBMVVkRHdFQi93UUVBd0lGb0RBZEJnTlZIU1VFRmpBVUJnZ3JCZ0VGQlFjREFRWUlLd1lCQlFVSEF3SXcKREFZRFZSMFRBUUgvQkFJd0FEQWZCZ05WSFNNRUdEQVdnQlJpeVdPK01LQ3JJWnhDQlZVb3IwMnY3STdabGpBegpCZ05WSFJFRUxEQXFnaEpqYjNKbExtaGhjbUp2Y2k1a2IyMWhhVzZDRkc1dmRHRnllUzVvWVhKaWIzSXVaRzl0CllXbHVNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJpK3pFdFllS3A3VnRnamo1Vk9PMXUzRERLSmJFaXZYRFYKbG9ZeVRjQzdWRGg3YlF5UmhPVEtja2pJeGd6cFpubjUyczYvSHYxbkQ3MTk0UXFieE0yOGI1OEpxUEtvS0E4aQpWNHEzSE9xUGhlMlRoS1M4ZUt4anl4R0ZRYi8rQ2prdXNlTDV0bjVFbFM1SjBIRU93K2crZjk0WUdMMTFhdkNuCklnazdDbjQwTWovUUg1YTdhaC9lenNad3Y1aFJISHQrc2pyeE5TYWFTcXNjbHQ4bEV2M3VQWFVsdHBrRm1VY2IKc3hOM1lwRDg4d05LWElQSnU5bWpGMnFJUUdUYXBtME04a3lvTGZpMHlrdDlWM1I1cXoybVcwQWx5d3dob0NvMQpTTm9KbUJIOE42bnhqU1JPUzVJS01XU0RhYWtLc1ZVWGhiUjJRQVlyYVkvZ3YrWnhmZEdlCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0Ktls.key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBdnZ6NDdJeHdiVGQwUDhXL1p4MVE1WGkvSEVmS2oyZkcxZStPQ0RMb2psSWEvbmErClJaWkc2b0lTSGxrek1aNnE4d1poWXk0VUUxQkIvVTNYSitsbjRzYU5IK1ZZU0I3Q25hbmFiWW5DVkp2Y091d1IKdy81WjVCc1NVK3hOeVNLS3d3RjdIS3A2QXZJSHNabmlPOWh4TDVFSkY0Zm5rR2NwUGZWSjNQTzdMY3B4ZDBVcApCdHpycmJwVlk1dXg5ZmN2SkljRXhCU2N2ZmhsZjd1Y0dKWEgwWWtMa2t0SVFQL3lnUXpDQWMvUG82NDJ6VUJqCjRNUVhlWkVUNlo0cWlrS1p3OFV0Rk8rbWZmQVNwT09xZjJKZ3AyYmlOMkdVakdOODE5MUw1NU1BUUY3aFBzUWoKNGNJNmhFVnM5N1ZrV3IvWSs3cTdxMjZGZ0NSMWNKemVQUGNablFJREFRQUJBb0lCQUEyOEVZYXBYdGhyMlg5UQprRUF4R29VMkZzQkk1V2RFdGtiaVVGdUVsYmJPYnNTSDg0ejdqcVFFc0pyZ0hVOWZNcm9CWm5XRWRjc2h4VzBhCjhQTWMxT3k1REtNVGtqSzFpNWRkQktsRjgrU3p4TFcwRGw4QzVxSUIxR0tXcHMxbEg0ZUFqc0x6KzR4RXJZcjMKSHc3Z3ROc1AyaENJSkgzOC9UbEliNFJsUHJ1L0tIV2p1SzJQWWQweFdzdDBtWVZBMytpYW9jWWJXTk1BV3p5cAo0WmR0c1Via2MrU3JPVUIvM2hrNE9iNUtkbUM3RE5aMTZJTnlSNGdpdXliSFI0Wk9JaHh1NGsyenFhVDBoU2J5CjREaUEyNytsczNWcGxETHhmanlKUVQyMEZHVW5jNTllb1pzdzBUeHZSUjM5cEFJQW9IdUhlL0ZCQW1RRnhZSEEKMS9qNUp5RUNnWUVBMnFnVDd4Sms4WFNLcStwenhDSTV4TlpTRVp0OU5WeVFZUUFzM25ubkxhemp0UUUwSGJGeQpWbk9ja1h6ZFd4Wnd5WUtUTktSb0ViSGE3V0NUVmd0T3RrR3VJWCtnenprQTRQaEI1YmhFNmp2bUhzS0laQkkvCmN5Z1lvMENZY0dZTGl5YnpRV0VLTzRPRHg1TlVxZmY2OTVRVVRGek1ORE10eSsvRnozc3lYNVVDZ1lFQTM1c3oKUmZHa0Qzd1lDTSszbWR3VldOZnRjbkNBYmw1YXppM3BWRG4yR2JRNC9GYVZkYVNmT3Z0Vk1oRUZpZkZGS2RzRApSNXJ3SmdXNmJSRUg0V1JOSEQxUzZ2WHc5ZnJJRkZFMnBRa2JZb01iV0tsUlFMNTlSYlRMQnEzQVJnV3Q4MXBYCmJtOXNNeFNBS2NBaVF6dGFxYXFTVDhOQjBHY1U0MjZSK3ZEYTcra0NnWUVBbFVRMlQzZE1KN1Q2VVgvOHhZRGoKMU9iR0liVDIxYTI1OWk0TGMxamVvMFNxTWM5L2gyR3lmeWZ2VXpaZFdpaEltSmVsN0VMcnRHQ281bkdPUXlmMgo5TjZEZytTL241YjNiWnlzUjZqeWlzQ1hTSnBlUjRwWmZFQjhDVDQ0a2twblNQZ3ZDWXU0VTRabE9LSHdJeFBoClpJL1hCNFkxOU5DWFMrZ0VMcTZZWmIwQ2dZRUExVVpsNEVlN2tOMXUyekNzVVQ5K3lPK2pWaTBDQXNOU1h5ZWgKVHFtK0Z3UXorbExuV1g4OE5QaTJhUHVkU2RYcmZ5R2JmamZFNks4OEFuMWxBOUUwVDBRYWkyc3JlcUxKSmIvVQpuQk1Vb0tDbWU2bDdpNEpsWUJBeU9kdU44ZnZHejc4U2Q0NGxLSTljTXZaRWQ5WHNBcnBqdFZwcXNza3ZQa0lmClVMZGNTOGtDZ1lCejZ2QlJqc2VlZnpySXVSRjR5UWorcUhZM2cxYXR2VDBMYlh5MWVvTk5RdW92ZXhoZmduTHoKangyVVB0Z29VMjlrMUpOWWhzMjByWVBhS1NDa2c5ZjNqUk1NNmZOc2M4QWQ5RFNJa3FaeS9rQkI0YjJPMDdKdwo2cVBRZEhyemJDK0Q2VUZhakFJMEFoVVV1WUZXbFRPaW5uLzlEc0p4Ky9CbEZSQzJFZTJtNEE9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
转码
[root@k8s-master harbor]# echo 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFekNDQWZ1Z0F3SUJBZ0lRY3d4c0NtVmNPV2RpZWJTSVREcU9kakFOQmdrcWhraUc5dzBCQVFzRkFEQVUKTVJJd0VBWURWUVFERXdsb1lYSmliM0l0WTJFd0hoY05Nakl4TWpJNE1EWXdOVE13V2hjTk1qTXhNakk0TURZdwpOVE13V2pBVU1SSXdFQVlEVlFRREV3bG9ZWEppYjNJdFkyRXdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCCkR3QXdnZ0VLQW9JQkFRQ3g0YVBFbkJ5K2xicElsSytyZmlsdndIWkJRMzM1a3lxMThKWHhpcDl1SFQ0VkExUzEKbG45anVtQUR4Smk4QnhWZStmVlBDWW5UeGJoNWJua3pqdm9XekptOTlMRUY5UDJPdW5VZnZCWGt6VFZEK0o1TgpBZjhyMmtqMFRTck5yZVVKL3JwR01mT1V1MFRSSjg2N2g1WHg1aEpBeDNlWlk4LzRFWmZQOVFGRlFsVGRxRDFBCmlkUkRydjFCSGJIajNxVjU3ckI4L3VxMVJ1a2hjQUZodlZXKzMvdmNkelBaS0hjc0c4d3VhNml2ekExK096a1gKTFA2K2wvcnlPektWWG9kU3ZHV1RmVzJpSlFEUjJqSWliR1loSUF3SDIrZlU4MVJrdGNsS050b2hZYW5aYjdOdQpTcTlzOFB1b3Y2V1JDMWpwRUd6cjhtR2hKTEhtcHJMNlRXeDdBZ01CQUFHallUQmZNQTRHQTFVZER3RUIvd1FFCkF3SUNwREFkQmdOVkhTVUVGakFVQmdnckJnRUZCUWNEQVFZSUt3WUJCUVVIQXdJd0R3WURWUjBUQVFIL0JBVXcKQXdFQi96QWRCZ05WSFE0RUZnUVVZc2xqdmpDZ3F5R2NRZ1ZWS0s5TnIreU8yWll3RFFZSktvWklodmNOQVFFTApCUUFEZ2dFQkFKd1NkMkFuMmpEdHp6NUIvOEl2UzliamxVN3hLVlR0M1pWWkc0N0R5aFJXTmRpVEs1NFF3dFdhCmtCYUZrV2xTQWJaUk5iOWwyak85NEFMelRCb0ttVU04QTBlZFBQbXdvTFFaZlV2NVp6SCtGM05PenNVT1I5dFIKbnhva3lkSU14WUlpQWp1YThudXRLK0g2TklsRDRiVVA1MVZSNFJsNjBhdHhSWi81OW1HMWw1ZUNVSWtObVJyaQpwY2FQTjBBa3NzR2dCNmpxK3VXMXhPU0l1M21idVl3amdzTEpqWTFkNkNuS1IvQ1RRZnRBKzB2NFNVRTJBTFRGClg0TEFRVXFsM0F2NU8wWkxFQXAzNE5aR0xGNCtnRmJuYi84MmdYTlBuY2IzQkxOR0tKVE5zL0RGTTBkYlgwajcKY3VIMHQvT01oV2YyZ084eTNBMDB6dkIzcTVZSkIwdz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=' | base64 -d > ca.crt
-----BEGIN CERTIFICATE-----
MIIDEzCCAfugAwIBAgIQcwxsCmVcOWdiebSITDqOdjANBgkqhkiG9w0BAQsFADAU
MRIwEAYDVQQDEwloYXJib3ItY2EwHhcNMjIxMjI4MDYwNTMwWhcNMjMxMjI4MDYw
NTMwWjAUMRIwEAYDVQQDEwloYXJib3ItY2EwggEiMA0GCSqGSIb3DQEBAQUAA4IB
DwAwggEKAoIBAQCx4aPEnBy+lbpIlK+rfilvwHZBQ335kyq18JXxip9uHT4VA1S1
ln9jumADxJi8BxVe+fVPCYnTxbh5bnkzjvoWzJm99LEF9P2OunUfvBXkzTVD+J5N
Af8r2kj0TSrNreUJ/rpGMfOUu0TRJ867h5Xx5hJAx3eZY8/4EZfP9QFFQlTdqD1A
idRDrv1BHbHj3qV57rB8/uq1RukhcAFhvVW+3/vcdzPZKHcsG8wua6ivzA1+OzkX
LP6+l/ryOzKVXodSvGWTfW2iJQDR2jIibGYhIAwH2+fU81RktclKNtohYanZb7Nu
Sq9s8Puov6WRC1jpEGzr8mGhJLHmprL6TWx7AgMBAAGjYTBfMA4GA1UdDwEB/wQE
AwICpDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwDwYDVR0TAQH/BAUw
AwEB/zAdBgNVHQ4EFgQUYsljvjCgqyGcQgVVKK9Nr+yO2ZYwDQYJKoZIhvcNAQEL
BQADggEBAJwSd2An2jDtzz5B/8IvS9bjlU7xKVTt3ZVZG47DyhRWNdiTK54QwtWa
kBaFkWlSAbZRNb9l2jO94ALzTBoKmUM8A0edPPmwoLQZfUv5ZzH+F3NOzsUOR9tR
nxokydIMxYIiAjua8nutK+H6NIlD4bUP51VR4Rl60atxRZ/59mG1l5eCUIkNmRri
pcaPN0AkssGgB6jq+uW1xOSIu3mbuYwjgsLJjY1d6CnKR/CTQftA+0v4SUE2ALTF
X4LAQUql3Av5O0ZLEAp34NZGLF4+gFbnb/82gXNPncb3BLNGKJTNs/DFM0dbX0j7
cuH0t/OMhWf2gO8y3A00zvB3q5YJB0w=
-----END CERTIFICATE-----
将证书保存在docker目录下，创建根域名相同的子目录，这一步建议所有机器都执行
[root@k8s-master harbor]# mkdir -p /etc/docker/certs.d/core.harbor.domain
[root@k8s-master harbor]# cp ca.crt /etc/docker/certs.d/core.harbor.domain/
6.配置host
建议所有机器执行
将harbor域名解析到ingress-nginx的svc的clusterIP，实现内网使用harbor的功能
[root@k8s-master harbor]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller-admission-nginx ClusterIP 10.106.125.71 <none> 443/TCP 48d
ingress-nginx-controller-nginx NodePort 10.108.233.252 <none> 80:31203/TCP,443:32292/TCP 48d
[root@k8s-master harbor]# echo '10.108.233.252 core.harbor.domain' >> /etc/hosts
三.测试
1.浏览器访问harbor
修改本机host
47.92.*.* core.harbor.domain
查看ingress-nginx暴露的https协议端口，这里我们能看到https协议的端口是32292
[root@k8s-master harbor]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller-admission-nginx ClusterIP 10.106.125.71 <none> 443/TCP 48d
ingress-nginx-controller-nginx NodePort 10.108.233.252 <none> 80:31203/TCP,443:32292/TCP 48d
浏览器访问
2.服务器内部访问harbor
[root@k8s-master harbor]# docker login https://core.harbor.domain
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded
3.上传镜像测试
这里使用我另外一篇文章制作的docker images，文章详细记录了dockerfile制作nginx镜像的过程
https://blog.csdn.net/weixin_43866248/article/details/128411117
另，推送命令可以在harbor的项目管理界面找到
[root@k8s-master harbor]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
nginx-1.16.1 v1 507f86395522 5 days ago 471MB
centos7-base v1 c31fb99ef249 6 days ago 432MB
[root@k8s-master harbor]# docker tag nginx-1.16.1:v1 core.harbor.domain/test/nginx-1.16.1:v1
[root@k8s-master harbor]# docker tag centos7-base:v1 core.harbor.domain/test/centos7-base:v1
[root@k8s-master harbor]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
core.harbor.domain/test/nginx-1.16.1 v1 507f86395522 5 days ago 471MB
nginx-1.16.1 v1 507f86395522 5 days ago 471MB
core.harbor.domain/test/centos7-base v1 c31fb99ef249 6 days ago 432MB
centos7-base v1 c31fb99ef249 6 days ago 432MB
[root@k8s-master harbor]# docker push core.harbor.domain/test/nginx-1.16.1:v1
The push refers to repository [core.harbor.domain/test/nginx-1.16.1]
d2f532751821: Pushed
0049738c43da: Pushed
457ed1526068: Pushed
174f56854903: Pushed
v1: digest: sha256:94449ea711d91742ffe6ea07546cce43b4c26aabc809b1a6824f903bb60aa8c7 size: 1164
[root@k8s-master harbor]# docker push core.harbor.domain/test/centos7-base:v1
The push refers to repository [core.harbor.domain/test/centos7-base]
457ed1526068: Mounted from test/nginx-1.16.1
174f56854903: Mounted from test/nginx-1.16.1
v1: digest: sha256:d97f75eb40907e9a3b1b8624be318fb8722a3412ac536c510e838c3408643f23 size: 741
4.下载镜像测试
在k8s-node1节点下载镜像
[root@k8s-node1 ~]# docker login https://core.harbor.domain
Username: admin
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin Succeeded
[root@k8s-node1 ~]# docker pull core.harbor.domain/test/nginx-1.16.1:v1
v1: Pulling from test/nginx-1.16.1
Digest: sha256:94449ea711d91742ffe6ea07546cce43b4c26aabc809b1a6824f903bb60aa8c7
Status: Downloaded newer image for core.harbor.domain/test/nginx-1.16.1:v1
core.harbor.domain/test/nginx-1.16.1:v1
[root@k8s-node1 ~]# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
core.harbor.domain/test/nginx-1.16.1 v1 507f86395522 5 days ago 471MB
5.创建一个工作负载测试
注意yaml文件里有个imagePullSecrets的参数，这是因为我在harbor里面创建了个访问级别是私有的test项目
[root@k8s-master test-namespace]# vim nginx-test.yaml
apiVersion: apps/v1
kind: Deployment
metadata:labels:k8s.kuboard.cn/name: nginx-testname: nginx-testnamespace: test
spec:replicas: 1selector:matchLabels:k8s.kuboard.cn/name: nginx-teststrategy:rollingUpdate:maxSurge: 25%maxUnavailable: 25%type: RollingUpdatetemplate:metadata:labels:k8s.kuboard.cn/name: nginx-testspec:containers:- image: 'core.harbor.domain/test/nginx-1.16.1:v1'imagePullPolicy: Alwaysname: nginxdnsPolicy: ClusterFirstimagePullSecrets:- name: harbor-secret
[root@k8s-master test-namespace]# vim harbor-secret.yaml ## 这个里面是harbor的认证文件
apiVersion: v1
data:.dockerconfigjson: eyJhdXRocyI6eyJodHRwczovL2NvcmUuaGFyYm9yLmRvbWFpbiI6eyJ1c2VybmFtZSI6ImFkbWluIiwicGFzc3dvcmQiOiJIYXJib3IxMjM0NSIsImF1dGgiOiJZV1J0YVc0NlNHRnlZbTl5TVRJek5EVT0ifX19
immutable: false
kind: Secret
metadata:name: harbor-secretnamespace: test
type: kubernetes.io/dockerconfigjson
[root@k8s-master test-namespace]# kubectl apply -f harbor-secret.yaml
[root@k8s-master test-namespace]# kubectl apply -f nginx-test.yaml
[root@k8s-master test-namespace]# kubectl get pod -n test ## 创建成功
NAME READY STATUS RESTARTS AGE
nginx-test-67f899486-tc95g 1/1 Running 0 21m
如果文章有遗漏，请评论或者私信
感谢
Helm安装Harbor相关推荐
- k8s使用helm安装harbor
1. 创建证书(注意修改域名参数) #创建目录保存证书文件 mkdir /data/harborsecret && cd /data/harborsecret # 生成 CA 证书私钥 ...
- Kubernetes - - k8s - v1.12.3 使用Helm安装harbor
1,Helm 介绍 核心术语: Chart:一个helm程序包 Repository:Charts仓库,https/http 服务器 Release:特定的Chart部署与目标集群上的一个实例 Cha ...
- 在kubernetes集群用helm离线安装harbor
背景说明 在公司内部局域网环境kubernetes集群(未连接互联网)通过helm离线安装harbor 实施步骤 一.kubernetes集群安装helm(已安装的直接跳过此节) 1. 关于helm ...
- kubernetes之helm部署harbor
安装 helm Helm致力于成为k8s集群的应用包管理工具,希望像linux 系统的RPM DPKG那样成功:确实在k8s上部署复杂一点的应用很麻烦,需要管理很多yaml文件(configmap,c ...
- helm部署harbor
helm部署harbor.chart 最新版本:1.5.0 namespace: public-service-ns.yaml apiVersion: v1 kind: Namespace metad ...
- helm3安装harbor【搭建NFS，用NFS创建PVC/PV供Harbor持久化，Harbor使用 nodePort 暴露方式提供访问】
一.安装nfs-server k8s-master01信息[提供nfs存储的机器] 公网IP:120.55.76.34 私网IP:172.30.125.99 未来的样子 nfs: server: 17 ...
- 简单安装Harbor私有镜像仓库
安装准备 Docker 17.06.0-ce+ Docker-compose 1.18.0+ 安装Docker 安装工具 yum install -y yum-utils device-mapper- ...
- K8S+Helm 安装 Jupyterhub
准备配置文件 生成随机数 openssl rand -hex 32 # 8fc2826e9ce6930ec26c9fd541c0620b448a947357edbdf9647516af16bbc798 ...
- helm安装_Helm部署和体验jenkins
运行在Kubernetes上的Jenkins 下图来自rancher官方博客,在kubernetes环境下,jenkins任务被交给各个pod执行,这些pod在需要时被创建,任务结束后被销毁,这样既能 ...
最新文章
- STM32学习笔记9（SysTick滴答时钟）
- 深度学习NCHW和NHWC数据格式（由三维数据转换成一维数据的遍历方式）
- 【深度学习】揭秘2021抖音和快手APP图像修复背后的核心技术，毫无ps痕迹
- Serverless在游戏运营行业进行数据采集分析的最佳实践 链接：
- 谈Apache OFbiz 会员模块表结构设计
- 在银行存100万，如果银行倒闭，超过50万部分是不是拿不回来？
- ps批量操作图片尺寸（一键执行） - 教程篇
- Java对象初始化执行顺序
- 初识shardingsphere
- OC开发_Storyboard——绘制和视图
- html判断字段不为空,js里是否为空字符串的判断
- 推荐一款Linux服务器连接工具FinalShell
- 短视频自媒体成功的秘诀就一个字：真
- WM_TIMER消息在线程被阻塞时的系统处理
- 自学python-自学python编程的方法路线
- C++使用opencv判断一个点是否在多边形之内
- 30分钟学习掌握springmvc、SSM
- Python编程 whl文件安装库
- html中如何做背景渐变,html如何设置背景径向渐变
- ipv6正则表达式 java_正则表达式，匹配所有有效格式的IPv6地址
热门文章
- Android压缩包下载解压
- gmail头像,字母头像（如Gmail Android最佳做法）
- ar 微信小程序_微信小程序开放AR功能，全面提升交互体验
- 微信小程序各门类需申请资质
- 华为扩大内存代码_华为畅享6内存扩大拨号代码 | 手游网游页游攻略大全
- 蓝牙BLE芯片PHY6222之烧录以及调试
- injected stylesheet注入样式导致el-button内文字为空白
- oracle 怎么看监听文件,【学习笔记】Oracle11G关于监听文件位置与监听文件大小限制...
- 应用程序无法开机自启动
- c语言如何实现人民币转换编程,C语言成序设计实现人民币小写金额与大写金额的转换.docx...