在 Kubernetes 中部署高可用 Harbor 镜像仓库

该文章随时会有校正更新，公众号无法更新，欢迎订阅博客查看最新内容：https://fuckcloudnative.io

前言

系统环境：

kubernetes 版本：1.18.10
Harbor Chart 版本：1.5.2
Harbor 版本：2.1.2
Helm 版本：3.3.4
持久化存储驱动：Ceph RBD

1. Harbor 简介

简介

Harbor 是一个开放源代码容器镜像注册表，可通过基于角色权限的访问控制来管理镜像，还能扫描镜像中的漏洞并将映像签名为受信任。Harbor 是 CNCF 孵化项目，可提供合规性，性能和互操作性，以帮助跨 Kubernetes 和 Docker 等云原生计算平台持续，安全地管理镜像。

特性

管理：多租户、可扩展
安全：安全和漏洞分析、内容签名与验证

2. 创建自定义证书

安装 Harbor 我们会默认使用 HTTPS 协议，需要 TLS 证书，如果我们没用自己设定自定义证书文件，那么 Harbor 将自动创建证书文件，不过这个有效期只有一年时间，所以这里我们生成自签名证书，为了避免频繁修改证书，将证书有效期为 100 年，操作如下：

安装 cfssl

fssl 是 CloudFlare 开源的一款 PKI/TLS 工具,cfssl 包含一个命令行工具和一个用于签名，验证并且捆绑 TLS 证书的HTTP API服务,使用 Go 语言编写.

github: https://github.com/cloudflare/cfssl^[1]

下载地址: https://pkg.cfssl.org/^[2]

macOS 安装步骤：

????  → brew install cfssl

通用安装方式：

????  → wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 -O /usr/local/bin/cfssl
????  → wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 -O /usr/local/bin/cfssljson
????  → wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 -O /usr/local/bin/cfssl-certinfo
????  → chmod +x /usr/local/bin/cfssl*

获取默认配置

????  → cfssl print-defaults config > ca-config.json
????  → cfssl print-defaults csr > ca-csr.json

生成 CA 证书

将ca-config.json内容修改为：

{"signing": {"default": {"expiry": "876000h"},"profiles": {"harbor": {"expiry": "876000h","usages": ["signing","key encipherment","server auth"]}}}
}

修改ca-csr.json文件内容为：

{"CN": "CA","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","ST": "hangzhou","L": "hangzhou","O": "harbor","OU": "System"}]
}

修改好配置文件后,接下来就可以生成 CA 证书了：

????  → cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2020/12/30 00:45:55 [INFO] generating a new CA key and certificate from CSR
2020/12/30 00:45:55 [INFO] generate received request
2020/12/30 00:45:55 [INFO] received CSR
2020/12/30 00:45:55 [INFO] generating key: rsa-2048
2020/12/30 00:45:56 [INFO] encoded CSR
2020/12/30 00:45:56 [INFO] signed certificate with serial number 529798847867094212963042958391637272775966762165

此时目录下会出现三个文件：

????  → tree
├── ca-config.json #这是刚才的json
├── ca.csr
├── ca-csr.json    #这也是刚才申请证书的json
├── ca-key.pem
├── ca.pem

这样我们就生成了:

根证书文件: ca.pem
根证书私钥: ca-key.pem
根证书申请文件: ca.csr (csr 是不是 client ssl request?)

签发证书

创建harbor-csr.json,内容为：

{"CN": "harbor","hosts": ["example.net","*.example.net"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "US","ST": "CA","L": "San Francisco","O": "harbor","OU": "System"}]
}

使用之前的 CA 证书签发 harbor 证书：

????  → cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=harbor harbor-csr.json | cfssljson -bare harbor
2020/12/30 00:50:31 [INFO] generate received request
2020/12/30 00:50:31 [INFO] received CSR
2020/12/30 00:50:31 [INFO] generating key: rsa-2048
2020/12/30 00:50:31 [INFO] encoded CSR
2020/12/30 00:50:31 [INFO] signed certificate with serial number 372641098655462687944401141126722021767151134362

此时目录下会多几个文件：

????  → tree -L 1
├── etcd.csr
├── etcd-csr.json
├── etcd-key.pem
├── etcd.pem

至此，harbor 的证书生成完成。

生成 Secret 资源

创建 Kubernetes 的 Secret 资源，且将证书文件导入：

- n：指定创建资源的 Namespace
--from-file：指定要导入的文件地址

????  → kubectl create ns harbor
????  → kubectl -n harbor create secret generic harbor-tls --from-file=tls.crt=harbor.pem --from-file=tls.key=harbor-key.pem --from-file=ca.crt=ca.pem

查看是否创建成功：

????  → kubectl -n harbor get secret harbor-tls
NAME         TYPE     DATA   AGE
harbor-tls   Opaque   3      1m

3. 使用 Ceph S3 为 Harbor chart 提供后端存储

创建 radosgw

如果你是通过 ceph-deploy 部署的，可以通过以下步骤创建 radosgw：

先安装 radosgw：

????  → ceph-deploy install --rgw 172.16.7.1 172.16.7.2 172.16.7.3

然后创建 radosgw：

????  → ceph-deploy rgw create 172.16.7.1 172.16.7.2 172.16.7.3

如果你是通过 cephadm 部署的，可以通过以下步骤创建 radosgw：

cephadm 将 radosgw 部署为管理特定领域和区域的守护程序的集合。例如，要在 172.16.7.1 上部署 1 个服务于 mytest 领域和 myzone 区域的 rgw 守护程序：

#如果尚未创建领域，请首先创建一个领域：
????  → radosgw-admin realm create --rgw-realm=mytest --default#接下来创建一个新的区域组：
????  → radosgw-admin zonegroup create --rgw-zonegroup=myzg --master --default#接下来创建一个区域：
????  → radosgw-admin zone create --rgw-zonegroup=myzg --rgw-zone=myzone --master --default#为特定领域和区域部署一组radosgw守护程序：
????  → ceph orch apply rgw mytest myzone --placement="1 172.16.7.1"

查看服务状态：

????  → ceph orch ls|grep rgw
rgw.mytest.myzone      1/1  5m ago     7w   count:1 k8s01  docker.io/ceph/ceph:v15     4405f6339e35

测试服务是否正常：

????  → curl -s http://172.16.7.1

正常返回如下数据：

<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets>
</ListAllMyBucketsResult>

查看 zonegroup：

????  → radosgw-admin zonegroup get
{"id": "ed34ba6e-7089-4b7f-91c4-82fc856fc16c","name": "myzg","api_name": "myzg","is_master": "true","endpoints": [],"hostnames": [],"hostnames_s3website": [],"master_zone": "650e7cca-aacb-4610-a589-acd605d53d23","zones": [{"id": "650e7cca-aacb-4610-a589-acd605d53d23","name": "myzone","endpoints": [],"log_meta": "false","log_data": "false","bucket_index_max_shards": 11,"read_only": "false","tier_type": "","sync_from_all": "true","sync_from": [],"redirect_zone": ""}],"placement_targets": [{"name": "default-placement","tags": [],"storage_classes": ["STANDARD"]}],"default_placement": "default-placement","realm_id": "e63c234c-e069-4a0d-866d-1ebdc69ec5fe","sync_policy": {"groups": []}
}

Create Auth Key

????  → ceph auth get-or-create client.radosgw.gateway osd 'allow rwx' mon 'allow rwx' -o /etc/ceph/ceph.client.radosgw.keyring

分发 /etc/ceph/ceph.client.radosgw.keyring 到其它 radosgw 节点。

创建对象存储用户和访问凭证

Create a radosgw user for s3 access

????  → radosgw-admin user create --uid="harbor" --display-name="Harbor Registry"

Create a swift user

????  → adosgw-admin subuser create --uid=harbor --subuser=harbor:swift --access=full

Create Secret Key

????  → radosgw-admin key create --subuser=harbor:swift --key-type=swift --gen-secret

记住 keys 字段中的 access_key & secret_key

创建存储桶（bucket）

首先需要安装 awscli：

????  → pip3 install awscli  -i https://pypi.tuna.tsinghua.edu.cn/simple

查看秘钥：

????  → radosgw-admin user info --uid="harbor"|jq .keys
[{"user": "harbor","access_key": "VGZQY32LMFQOQPVNTDSJ","secret_key": "YZMMYqoy1ypHaqGOUfwLvdAj9A731iDYDjYqwkU5"}
]

配置 awscli：

????  → aws configure --profile=ceph
AWS Access Key ID [None]: VGZQY32LMFQOQPVNTDSJ
AWS Secret Access Key [None]: YZMMYqoy1ypHaqGOUfwLvdAj9A731iDYDjYqwkU5
Default region name [None]:
Default output format [None]: json

配置完成后，凭证将会存储到 ~/.aws/credentials：

????  → cat ~/.aws/credentials
[ceph]
aws_access_key_id = VGZQY32LMFQOQPVNTDSJ
aws_secret_access_key = YZMMYqoy1ypHaqGOUfwLvdAj9A731iDYDjYqwkU5

配置将会存储到 ~/.aws/config：

????  → cat ~/.aws/config
[profile ceph]
region = cn-hangzhou-1
output = json

创建存储桶（bucket）：

????  → aws --profile=ceph --endpoint=http://172.16.7.1 s3api create-bucket --bucket harbor

查看存储桶（bucket）列表：

????  → radosgw-admin bucket list
["harbor"
]

查看存储桶状态：

????  → radosgw-admin bucket stats
[{"bucket": "harbor","num_shards": 11,"tenant": "","zonegroup": "ed34ba6e-7089-4b7f-91c4-82fc856fc16c","placement_rule": "default-placement","explicit_placement": {"data_pool": "","data_extra_pool": "","index_pool": ""},"id": "650e7cca-aacb-4610-a589-acd605d53d23.194274.1","marker": "650e7cca-aacb-4610-a589-acd605d53d23.194274.1","index_type": "Normal","owner": "harbor","ver": "0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver": "0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime": "2020-12-29T17:19:02.481567Z","creation_time": "2020-12-29T17:18:58.940915Z","max_marker": "0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage": {},"bucket_quota": {"enabled": false,"check_on_raw": false,"max_size": -1,"max_size_kb": 0,"max_objects": -1}}
]

查看存储池状态

????  → rados df
POOL_NAME                    USED  OBJECTS  CLONES  COPIES  MISSING_ON_PRIMARY  UNFOUND  DEGRADED    RD_OPS       RD     WR_OPS       WR  USED COMPR  UNDER COMPR
.rgw.root                 2.3 MiB       13       0      39                   0        0         0       533  533 KiB         21   16 KiB         0 B          0 B
cache                         0 B        0       0       0                   0        0         0         0      0 B          0      0 B         0 B          0 B
device_health_metrics     3.2 MiB       18       0      54                   0        0         0       925  929 KiB        951  951 KiB         0 B          0 B
kubernetes                735 GiB    72646      99  217938                   0        0         0  48345148  242 GiB  283283048  7.3 TiB         0 B          0 B
myzone.rgw.buckets.index  8.6 MiB       11       0      33                   0        0         0        44   44 KiB         11      0 B         0 B          0 B
myzone.rgw.control            0 B        8       0      24                   0        0         0         0      0 B          0      0 B         0 B          0 B
myzone.rgw.log              6 MiB      206       0     618                   0        0         0   2188882  2.1 GiB    1457026   32 KiB         0 B          0 B
myzone.rgw.meta           960 KiB        6       0      18                   0        0         0        99   80 KiB         17    8 KiB         0 B          0 Btotal_objects    72908
total_used       745 GiB
total_avail      87 TiB
total_space      88 TiB

3. 设置 Harbor 配置清单

由于我们需要通过 Helm 安装 Harbor 仓库，需要提前创建 Harbor Chart 的配置清单文件，里面是对要创建的应用 Harbor 进行一系列参数配置，由于参数过多，关于都有 Harbor Chart 都能够配置哪些参数这里就不一一罗列，可以通过访问 Harbor-helm 的 Github 地址^[3] 进行了解。

下面描述下，需要的一些配置参数：

values.yaml

#入口配置，我只在内网使用，所以直接使用 cluserIP
expose:type: clusterIPtls:### 是否启用 https 协议enabled: truecertSource: secretauto:# The common name used to generate the certificate, it's necessary# when the type isn't "ingress"commonName: "harbor.example.net"secret:# The name of secret which contains keys named:# "tls.crt" - the certificate# "tls.key" - the private keysecretName: "harbor-tls"# The name of secret which contains keys named:# "tls.crt" - the certificate# "tls.key" - the private key# Only needed when the "expose.type" is "ingress".notarySecretName: ""## 如果Harbor部署在代理后，将其设置为代理的URL
externalURL: https://harbor.example.net### Harbor 各个组件的持久化配置，并将 storageClass 设置为集群默认的 storageClass
persistence:enabled: true# Setting it to "keep" to avoid removing PVCs during a helm delete# operation. Leaving it empty will delete PVCs after the chart deleted# (this does not apply for PVCs that are created for internal database# and redis components, i.e. they are never deleted automatically)resourcePolicy: "keep"persistentVolumeClaim:registry:# Use the existing PVC which must be created manually before bound,# and specify the "subPath" if the PVC is shared with other componentsexistingClaim: ""# Specify the "storageClass" used to provision the volume. Or the default# StorageClass will be used(the default).# Set it to "-" to disable dynamic provisioningstorageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 100Gichartmuseum:existingClaim: ""storageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 5Gijobservice:existingClaim: ""storageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 5Gi# If external database is used, the following settings for database will# be ignoreddatabase:existingClaim: ""storageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 5Gi# If external Redis is used, the following settings for Redis will# be ignoredredis:existingClaim: ""storageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 5Gitrivy:existingClaim: ""storageClass: "csi-rbd-sc"subPath: ""accessMode: ReadWriteOncesize: 5Gi### 默认用户名 admin 的密码配置，注意：密码中一定要包含大小写字母与数字
harborAdminPassword: "Mydlq123456"### 设置日志级别
logLevel: info#各个组件 CPU & Memory 资源相关配置
nginx:resources:requests:memory: 256Micpu: 500m
portal:resources:requests:memory: 256Micpu: 500m
core:resources:requests:memory: 256Micpu: 1000m
jobservice:resources:requests:memory: 256Micpu: 500m
registry:registry:resources:requests:memory: 256Micpu: 500mcontroller:resources:requests:memory: 256Micpu: 500m
clair:clair:resources:requests:memory: 256Micpu: 500madapter:resources:requests:memory: 256Micpu: 500m
notary:server:resources:requests:memory: 256Micpu: 500msigner:resources:requests:memory: 256Micpu: 500m
database:internal:resources:requests:memory: 256Micpu: 500m
redis:internal:resources:requests:memory: 256Micpu: 500m
trivy:enabled: trueresources:requests:cpu: 200mmemory: 512Milimits:cpu: 1000mmemory: 1024Mi#开启 chartmuseum，使 Harbor 能够存储 Helm 的 chart
chartmuseum:enabled: trueresources:requests:memory: 256Micpu: 500mimageChartStorage:# Specify whether to disable `redirect` for images and chart storage, for# backends which not supported it (such as using minio for `s3` storage type), please disable# it. To disable redirects, simply set `disableredirect` to `true` instead.# Refer to# https://github.com/docker/distribution/blob/master/docs/configuration.md#redirect# for the detail.disableredirect: false# Specify the "caBundleSecretName" if the storage service uses a self-signed certificate.# The secret must contain keys named "ca.crt" which will be injected into the trust store# of registry's and chartmuseum's containers.# caBundleSecretName:# Specify the type of storage: "filesystem", "azure", "gcs", "s3", "swift",# "oss" and fill the information needed in the corresponding p. The type# must be "filesystem" if you want to use persistent volumes for registry# and chartmuseumtype: s3s3:region: cn-hangzhou-1bucket: harboraccesskey: VGZQY32LMFQOQPVNTDSJsecretkey: YZMMYqoy1ypHaqGOUfwLvdAj9A731iDYDjYqwkU5regionendpoint: http://172.16.7.1#encrypt: false#keyid: mykeyidsecure: false#skipverify: false#v4auth: true#chunksize: "5242880"#rootdirectory: /s3/object/name/prefix#storageclass: STANDARD#multipartcopychunksize: "33554432"#multipartcopymaxconcurrency: 100#multipartcopythresholdsize: "33554432"

4. 安装 Harbor

添加 Helm 仓库

????  → helm repo add harbor https://helm.goharbor.io

部署 Harbor

????  → helm install harbor harbor/harbor -f values.yaml -n harbor

查看应用是否部署完成

????  → kubectl -n harbor get pod
NAME                                          READY   STATUS    RESTARTS   AGE
harbor-harbor-chartmuseum-55fb975fbd-74vnh    1/1     Running   0          3m
harbor-harbor-clair-695c7f9c69-7gpkh          2/2     Running   0          3m
harbor-harbor-core-687cfb49b6-zmwxr           1/1     Running   0          3m
harbor-harbor-database-0                      1/1     Running   0          3m
harbor-harbor-jobservice-88994b9b7-684vb      1/1     Running   0          3m
harbor-harbor-nginx-6758559548-x9pq6          1/1     Running   0          3m
harbor-harbor-notary-server-6d55b785f-6jsq9   1/1     Running   0          3m
harbor-harbor-notary-signer-9696cbdd8-8tfw9   1/1     Running   0          3m
harbor-harbor-portal-6f474574c4-8jzh2         1/1     Running   0          3m
harbor-harbor-redis-0                         1/1     Running   0          3m
harbor-harbor-registry-5b6cbfb4cf-42fm9       2/2     Running   0          3m
harbor-harbor-trivy-0                         1/1     Running   0          3m

Host 配置域名

接下来配置 Hosts，客户端想通过域名访问服务，必须要进行 DNS 解析，由于这里没有 DNS 服务器进行域名解析，所以修改 hosts 文件将 Harbor 指定 clusterIP 和自定义 host 绑定。首先查看 nginx 的 clusterIP：

????  → kubectl -n harbor get svc harbor-harbor-nginx
NAME                  TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
harbor-harbor-nginx   ClusterIP   10.109.50.142   <none>        80/TCP,443/TCP   22h

打开主机的 Hosts 配置文件，往其加入下面配置：

10.109.50.142 harbor.example.net

如果想在集群外访问，建议将 Service nginx 的 type 改为 nodePort 或者通过 ingress 来代理。当然，如果你在集群外能够直接访问 clusterIP，那更好。

输入地址 https://harbor.example.net 访问 Harbor 仓库。

用户：admin
密码：Mydlq123456 (在安装配置中自定义的密码)

进入后可以看到 Harbor 的管理后台：

5. 服务器配置镜像仓库

对于 Containerd 来说，不能像 docker 一样 docker login 登录到镜像仓库，需要修改其配置文件来进行认证。/etc/containerd/config.toml 需要添加如下内容：

    [plugins."io.containerd.grpc.v1.cri".registry][plugins."io.containerd.grpc.v1.cri".registry.mirrors]...[plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.net".tls]insecure_skip_verify = true[plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.net".auth]username = "admin"password = "Mydlq123456"

由于 Harbor 是基于 Https 的，理论上需要提前配置 tls 证书，但可以通过 insecure_skip_verify 选项跳过证书认证。

当然，如果你想通过 Kubernetes 的 secret 来进行用户验证，配置还可以精简下：

    [plugins."io.containerd.grpc.v1.cri".registry][plugins."io.containerd.grpc.v1.cri".registry.mirrors]...[plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.net".tls]insecure_skip_verify = true

Kubernetes 集群使用 docker-registry 类型的 Secret 来通过镜像仓库的身份验证，进而拉取私有映像。所以需要创建 Secret，命名为 regcred：

????  → kubectl create secret docker-registry regcred \--docker-server=<你的镜像仓库服务器> \--docker-username=<你的用户名> \--docker-password=<你的密码> \--docker-email=<你的邮箱地址>

然后就可以在 Pod 中使用该 secret 来访问私有镜像仓库了，下面是一个示例 Pod 配置文件：

apiVersion: v1
kind: Pod
metadata:name: private-reg
spec:containers:- name: private-reg-containerimage: <your-private-image>imagePullSecrets:- name: regcred

如果你不嫌麻烦，想更安全一点，那就老老实实将 CA、证书和秘钥拷贝到所有节点的 /etc/ssl/certs/ 目录下。/etc/containerd/config.toml 需要添加的内容更多一点：

    [plugins."io.containerd.grpc.v1.cri".registry][plugins."io.containerd.grpc.v1.cri".registry.mirrors]...[plugins."io.containerd.grpc.v1.cri".registry.configs][plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.example.net".tls]ca_file = "/etc/ssl/certs/ca.pem"cert_file = "/etc/ssl/certs/harbor.pem"key_file  = "/etc/ssl/certs/harbor-key.pem"

至于 Docker 的配置方式，大家可以自己去搜一下，这里就跳过了，谁让它现在不受待见呢。

6. 测试功能

这里为了测试推送镜像，先下载一个用于测试的 helloworld 小镜像，然后推送到 harbor.example.net 仓库：

### 拉取 Helloworld 镜像
????  → ctr i pull bxsfpjcb.mirror.aliyuncs.com/library/hello-world:latest
bxsfpjcb.mirror.aliyuncs.com/library/hello-world:latest:                          resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec:    done           |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0e03bdcc26d7a9a57ef3b6f1bf1a210cff6239bff7c8cac72435984032851689:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 15.8s                                                                    total:  2.6 Ki (166.0 B/s)
unpacking linux/amd64 sha256:1a523af650137b8accdaed439c17d684df61ee4d74feac151b5b337bd29e7eec...
done### 将下载的镜像使用 tag 命令改变镜像名
????  → ctr i tag bxsfpjcb.mirror.aliyuncs.com/library/hello-world:latest harbor.example.net/library/hello-world:latest
harbor.example.net/library/hello-world:latest### 推送镜像到镜像仓库
????  → ctr i push --user admin:Mydlq123456 --platform linux/amd64 harbor.example.net/library/hello-world:latest
manifest-sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042: done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b:   done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0e03bdcc26d7a9a57ef3b6f1bf1a210cff6239bff7c8cac72435984032851689:    done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 2.2 s                                                                    total:  4.5 Ki (2.0 KiB/s)

镜像仓库中也能看到：

将之前的下载的镜像删除，然后测试从 harbor.example.net 下载镜像进行测试：

### 删除之前镜像
????  → ctr i rm harbor.example.net/library/hello-world:latest
????  → ctr i rm bxsfpjcb.mirror.aliyuncs.com/library/hello-world:latest### 测试从 harbor.example.net 下载新镜像
????  → ctr i pull harbor.example.net/library/hello-world:latest
harbor.example.net/library/hello-world:latest:                                   resolved       |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042: done           |++++++++++++++++++++++++++++++++++++++|
layer-sha256:0e03bdcc26d7a9a57ef3b6f1bf1a210cff6239bff7c8cac72435984032851689:    done           |++++++++++++++++++++++++++++++++++++++|
config-sha256:bf756fb1ae65adf866bd8c456593cd24beb6a0a061dedf42b26a993176745f6b:   done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 0.6 s                                                                    total:  525.0  (874.0 B/s)
unpacking linux/amd64 sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042...
done

参考

通过 Helm 搭建 Docker 镜像仓库 Harbor^[4]

参考资料

[1]

https://github.com/cloudflare/cfssl: https://github.com/cloudflare/cfssl

[2]

https://pkg.cfssl.org/: https://pkg.cfssl.org/

[3]

Harbor-helm 的 Github 地址: https://github.com/goharbor/harbor-helm

[4]

通过 Helm 搭建 Docker 镜像仓库 Harbor: http://www.mydlq.club/article/66/

你可能还喜欢

点击下方图片即可阅读

高性能 Nginx HTTPS 调优 - 如何为 HTTPS 提速 30%

云原生是一种信仰 ????

扫码关注公众号

后台回复◉k8s◉获取史上最方便快捷的 Kubernetes 高可用部署工具，只需一条命令，连 ssh 都不需要！

点击 "阅读原文" 获取更好的阅读体验！

❤️给个「在看」，是对我最大的支持❤️