Harbor中镜像清理

背景

项目的每日构建中，由于微服务比较多，每天会产生比较多的镜像，导致Harbor服务器很快被占满。所以要定期去清理不需要的镜像来释放存储空间。

使用的Harbor版本是2.5.0

镜像清理

镜像清理可以手动在Harbor页面上操作。根据Harbor官方文档，当我们删除某个镜像时，存储空间不会被真正的释放，必须要手动去执行垃圾清理来释放不再被引用的blobs文件。

页面操作

首先在页面删除不需要的镜像信息。
然后在系统管理->垃圾清理，选择立即清理垃圾，手动GC。
在立即清理右边有一个模拟运行，运行这个会打印出符合删除条件的blobs，并且会计算将会释放多少存储空间，不会真正的去删除文件，每次清理之前可以先执行一下模拟运行来防止误操作。
清理操作触发之后，在历史记录，查看任务日志可以看到释放了多少的存储空间，完成清理。

定时清理工具

上面的手动清理不够灵活，我们可以根据Harbor提供的restful接口编写定时清理镜像的程序。

Harbor的接口需要提供一个basic认证的信息。我这边根据我们的业务场景编写了一个定时清理的程序，具体场景：每天定时扫描Harbor上我们CI构建的项目，只保留最近10次的构建推包镜像，其余的全部删除。

代码参考如下：

/*** 定时扫描harbor中的项目 只保留最近10次构建的镜像信息** @author yuanzhihao* @since 2022/6/29*/
@Slf4j
public class Main {private static final String USERNAME = "admin";private static final String PASSWORD = "Harbor12345";// 默认保存镜像的最大数量private static final int DEFAULT_REMAIN_ARTIFACT_COUNT = 10;// 需要清理的项目列表private static final List<String> PROJECTS = Arrays.asList("common", "library");private static final String GET_PROJECTS_INFO_URL = "http://192.168.1.103/api/v2.0/projects/%s";private static final String GET_REPOSITORIES_URL = "http://192.168.1.103/api/v2.0/projects/%s/repositories?page_size=%s";// 根据镜像push时间降序private static final String GET_ARTIFACTS_URL = "http://192.168.1.103/api/v2.0/projects/%s/repositories/%s/artifacts?page_size=%s&sort=-push_time";private static final String DELETE_ARTIFACTS_URL = "http://192.168.1.103/api/v2.0/projects/%s/repositories/%s/artifacts/%s";private static final String GC_URL = "http://192.168.1.103/api/v2.0/system/gc/schedule";public static void main(String[] args) {ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor();// 每天定时清理executorService.scheduleWithFixedDelay(new Runnable() {@SneakyThrows@Overridepublic void run() {log.info("Garbage Collection Start");for (String project : PROJECTS) {final int repoCount = getRepoCount(project);final List<Repository> repositories = getRepositories(project, repoCount);for (Repository repository : repositories) {String repoName = repository.name.split("/")[1];final List<String> artifacts = getArtifacts(project, repoName, repository.artifact_count);// 只保留最近10次构建镜像final int size = artifacts.size();if (size > DEFAULT_REMAIN_ARTIFACT_COUNT) {log.info("Project [{}] Repo [{}] Need GC", project, repoName);for (int i = DEFAULT_REMAIN_ARTIFACT_COUNT; i < size; i++) {deleteSpecificArtifact(project, repoName, artifacts.get(i));log.info("Project [{}] Repo [{}] Artifact [{}] Delete Success", project, repoName, artifacts.get(i));}}}}// 最后执行GC释放空间garbageCollect();log.info("Garbage Collection End");}}, 0, 1, TimeUnit.DAYS);}// 根据项目名称获取仓库数量private static int getRepoCount(String project) throws IOException {String url = String.format(GET_PROJECTS_INFO_URL, project);final String result = httpRequest(url, HttpMethod.GET, null);return JsonParser.parseString(result).getAsJsonObject().get("repo_count").getAsInt();}// 获取指定项目下所有的仓库列表private static List<Repository> getRepositories(String project, int size) throws IOException {String url = String.format(GET_REPOSITORIES_URL, project, size);final String result = httpRequest(url, HttpMethod.GET, null);return new Gson().fromJson(result, new TypeToken<List<Repository>>() {}.getType());}// 获取指定镜像仓库中的镜像列表 按照镜像push的时间排序private static List<String> getArtifacts(String project, String repository, int size) throws IOException {String url = String.format(GET_ARTIFACTS_URL, project, repository, size);final String result = httpRequest(url, HttpMethod.GET, null);List<String> artifactList = new ArrayList<>();JsonParser.parseString(result).getAsJsonArray().forEach(item -> artifactList.add(item.getAsJsonObject().get("digest").getAsString()));return artifactList;}// 根据ArtifactId删除镜像private static void deleteSpecificArtifact(String project, String repository, String artifactId) throws IOException {String url = String.format(DELETE_ARTIFACTS_URL, project, repository, artifactId);httpRequest(url, HttpMethod.DELETE, null);}// 触发gcprivate static void garbageCollect() throws IOException {String requestParameters = "{\"parameters\":{\"delete_untagged\":false,\"dry_run\":false},\"schedule\":{\"type\":\"Manual\"}}";httpRequest(GC_URL, HttpMethod.POST, requestParameters);}private static String httpRequest(String url, HttpMethod method, String requestBody) throws IOException {HttpRequestBase request = getRequest(url, method);request.addHeader("authorization", getAuthorization());if (StringUtils.isNotEmpty(requestBody)) {StringEntity stringEntity = new StringEntity(requestBody, "UTF-8");stringEntity.setContentType("application/json;charset=UTF-8");((HttpEntityEnclosingRequestBase) request).setEntity(stringEntity);}try (CloseableHttpClient client = HttpClients.createDefault()) {try (CloseableHttpResponse response = client.execute(request)) {int statusCode = response.getStatusLine().getStatusCode();if (statusCode == HttpStatus.SC_OK || statusCode == HttpStatus.SC_CREATED) {return EntityUtils.toString(response.getEntity(), "UTF-8");}log.error("Http Request error, HttpStatus is [{}]", statusCode);}}return "";}private static String getAuthorization() {byte[] encodeAuth = Base64.getEncoder().encode((USERNAME + ":" + PASSWORD).getBytes());return "Basic " + new String(encodeAuth);}private static HttpRequestBase getRequest(String url, HttpMethod method) {switch (method) {case GET:return new HttpGet(url);case POST:return new HttpPost(url);case DELETE:return new HttpDelete(url);default:throw new IllegalArgumentException("Unsupported method");}}// 仓库信息static class Repository {int artifact_count;String creation_time;int id;String name;int project_id;int pull_count;String update_time;}enum HttpMethod {GET, POST, DELETE}
}

结语

参考：https://goharbor.io/docs/2.5.0/administration/garbage-collection

代码地址：https://github.com/yzh19961031/blogDemo/tree/master/harborImageClean

Harbor中镜像清理相关推荐

Harbor私有镜像仓库
Harbor简介 CA认证 https通过ssl的加密认证形式:要求http拥有加密认证证书才能转换成https. 私有镜像仓库需要启动https的模式 VMware的开源项目https://gith ...
Harbor私有镜像仓库（上）
Harbor私有镜像仓库(上) 链接:https://pan.baidu.com/s/1MAb0dllUwmoOk7TeVCZOVQ 提取码:ldt5 复制这段内容后打开百度网盘手机App,操作更方便 ...
docker搭建harbor私有镜像库
创建harbor私有镜像库一.部署准备: harbor软件包在部署节点上: 1)解压harbor的软件包将harbor目录下所有文件发送到/opt/目录下 tar zxvf harbor-offl ...
harbor的镜像复制功能使用教程
环境 192.168.0.83是harbor01, 192.168.0.84是harbor02 每个harbor的域名是 harbor.youwoyouqu.io 进行复制时,harbor02会通过配 ...
教你如何用Harbor 私有镜像仓库搭建
一.Harbor是什么? Docker有个形象的比喻叫集装箱,kubernetes是舵手,而Harbor是港湾,其实是用来保存容器镜像的仓库,企业使用docker.kubernetes时,一 ...
harbor企业级镜像仓库搭建
前言环境:centos 7.9 harbor-offline-installer-v2.4.3.tgz 下载harbor版本 harbor的官网:https://goharbor.io/,点下载按钮 ...
harbor https镜像仓库安装及原理概述
harbor https镜像仓库一键安装及原理概述(基于官网2023年架构) 1.简介 Harbor是VMware公司开源的企业级DockerRegistry项目,其目标是帮助用户迅速搭建一个企业级的 ...
搭建Harbor私有镜像仓库
Docker容器应用的开发和运行离不开可靠的镜像管理,部署一个私有的镜像仓库是十分有必要的.虽然Docker官方提供了docker-registry用于构建私有的镜像仓库.但它的功能没有Harbor丰 ...
cad镜像后标注尺寸数字反了_CAD中镜像文字后，文字反转了，可以这样轻松调整过来...
CAD制图时,"镜像"命令用得很频繁,很多图形绘制都要用到镜像,镜像命令可让我们迅速完成对称图形的绘制,其方便性无需多言,大家也很熟悉,但是,当文字"镜像"后, ...

Harbor中镜像清理

Harbor中镜像清理

背景

镜像清理

页面操作

定时清理工具

结语

Harbor中镜像清理相关推荐

最新文章

热门文章