1.RepositoriesModule概述

Sets up classes for Snapshot/Restore

1.1 snapshot概述

A snapshot is a backup taken from a running Elasticsearch cluster. You can take a snapshot of individual indices or of the entire cluster and store it in a repository on a shared filesystem, and there are plugins that support remote repositories on S3, HDFS, Azure, Google Cloud Storage and more.Snapshots are taken incrementally. This means that when creating a snapshot of an index Elasticsearch will avoid copying any data that is already stored in the repository as part of an earlier snapshot of the same index. Therefore it can be efficient to take snapshots of your cluster quite frequently.Snapshots can be restored into a running cluster via the restore API. When restoring an index it is possible to alter the name of the restored index as well as some of its settings, allowing a great deal of flexibility in how the snapshot and restore functionality can be used.

1.2 repository

You must register a snapshot repository before you can perform snapshot and restore operations. We recommend creating a new snapshot repository for each major version. The valid repository settings depend on the repository type.If you register same snapshot repository with multiple clusters, only one cluster should have write access to the repository. All other clusters connected to that repository should set the repository to readonly mode.

实例:备份

PUT /_snapshot/my_backup
{"type": "fs","settings": {"location": "my_backup_location"}
}

恢复

POST /_snapshot/my_backup/snapshot_1/_restore

删除

DELETE /_snapshot/my_backup/snapshot_1

监控

GET /_snapshot/my_backup/snapshot_1/_status

2.配置类BlobStoreRepository

    protected BlobStoreRepository(RepositoryMetaData metadata, Settings globalSettings, NamedXContentRegistry namedXContentRegistry) {super(globalSettings);this.metadata = metadata;this.namedXContentRegistry = namedXContentRegistry;snapshotRateLimiter = getRateLimiter(metadata.settings(), "max_snapshot_bytes_per_sec", new ByteSizeValue(40, ByteSizeUnit.MB));restoreRateLimiter = getRateLimiter(metadata.settings(), "max_restore_bytes_per_sec", new ByteSizeValue(40, ByteSizeUnit.MB));readOnly = metadata.settings().getAsBoolean("readonly", false);indexShardSnapshotFormat = new ChecksumBlobStoreFormat<>(SNAPSHOT_CODEC, SNAPSHOT_NAME_FORMAT,BlobStoreIndexShardSnapshot::fromXContent, namedXContentRegistry, isCompress());indexShardSnapshotsFormat = new ChecksumBlobStoreFormat<>(SNAPSHOT_INDEX_CODEC, SNAPSHOT_INDEX_NAME_FORMAT,BlobStoreIndexShardSnapshots::fromXContent, namedXContentRegistry, isCompress());ByteSizeValue chunkSize = chunkSize();if (chunkSize != null && chunkSize.getBytes() <= 0) {throw new IllegalArgumentException("the chunk size cannot be negative: [" + chunkSize + "]");}}

配置类的实现

3.重点类

3.1 SnapshotsService

创建snapshot

/*** Service responsible for creating snapshots* <p>* A typical snapshot creating process looks like this:* <ul>* <li>On the master node the {@link #createSnapshot(SnapshotRequest, CreateSnapshotListener)} is called and makes sure that no snapshots is currently running* and registers the new snapshot in cluster state</li>* <li>When cluster state is updated the {@link #beginSnapshot(ClusterState, SnapshotsInProgress.Entry, boolean, CreateSnapshotListener)} method* kicks in and initializes the snapshot in the repository and then populates list of shards that needs to be snapshotted in cluster state</li>* <li>Each data node is watching for these shards and when new shards scheduled for snapshotting appear in the cluster state, data nodes* start processing them through {@link SnapshotShardsService#processIndexShardSnapshots(ClusterChangedEvent)} method</li>* <li>Once shard snapshot is created data node updates state of the shard in the cluster state using the {@link SnapshotShardsService#sendSnapshotShardUpdate(Snapshot, ShardId, ShardSnapshotStatus)} method</li>* <li>When last shard is completed master node in {@link SnapshotShardsService#innerUpdateSnapshotState} method marks the snapshot as completed</li>* <li>After cluster state is updated, the {@link #endSnapshot(SnapshotsInProgress.Entry)} finalizes snapshot in the repository,* notifies all {@link #snapshotCompletionListeners} that snapshot is completed, and finally calls {@link #removeSnapshotFromClusterState(Snapshot, SnapshotInfo, Exception)} to remove snapshot from cluster state</li>* </ul>*/

3.2 RestoreService

/*** Service responsible for restoring snapshots* <p>* Restore operation is performed in several stages.* <p>* First {@link #restoreSnapshot(RestoreRequest, org.elasticsearch.action.ActionListener)}* method reads information about snapshot and metadata from repository. In update cluster state task it checks restore* preconditions, restores global state if needed, creates {@link RestoreInProgress} record with list of shards that needs* to be restored and adds this shard to the routing table using {@link RoutingTable.Builder#addAsRestore(IndexMetaData, SnapshotRecoverySource)}* method.* <p>* Individual shards are getting restored as part of normal recovery process in* {@link IndexShard#restoreFromRepository(Repository)} )}* method, which detects that shard should be restored from snapshot rather than recovered from gateway by looking* at the {@link ShardRouting#recoverySource()} property.* <p>* At the end of the successful restore process {@code RestoreService} calls {@link #cleanupRestoreState(ClusterChangedEvent)},* which removes {@link RestoreInProgress} when all shards are completed. In case of* restore failure a normal recovery fail-over process kicks in.*/

转载于:https://www.cnblogs.com/davidwang456/p/10138559.html

elasticSearch6源码分析(8)RepositoriesModule模块相关推荐

  1. elasticSearch6源码分析(5)gateway模块

    1.gateway概述 The local gateway module stores the cluster state and shard data across full cluster res ...

  2. elasticSearch6源码分析(4)indices模块

    1.indices概述 The indices module controls index-related settings that are globally managed for all ind ...

  3. elasticSearch6源码分析(3)cluster模块

    1. cluser概述 One of the main roles of the master is to decide which shards to allocate to which nodes ...

  4. elasticsearch源码分析之search模块(server端)

    elasticsearch源码分析之search模块(server端) 继续接着上一篇的来说啊,当client端将search的请求发送到某一个node之后,剩下的事情就是server端来处理了,具体 ...

  5. elasticsearch源码分析之search模块(client端)

    elasticsearch源码分析之search模块(client端) 注意,我这里所说的都是通过rest api来做的搜索,所以对于接收到请求的节点,我姑且将之称之为client端,其主要的功能我们 ...

  6. FreeCAD源码分析:FreeCADGui模块

    FreeCAD源码分析:FreeCADGui模块 济南友泉软件有限公司 FreeCADGui项目实现了界面操作.模型显示与交互等相关功能,项目构建生成FreeCAD(_d).dll动态链接库. Fre ...

  7. Python3.5源码分析-内建模块builtins初始化

    Python3源码分析 本文环境python3.5.2. 参考书籍<<Python源码剖析>> python官网 Python3模块初始化与加载 Python的模块分为内建的模 ...

  8. dubbo源码分析系列——dubbo-cluster模块源码分析

    2019独角兽企业重金招聘Python工程师标准>>> 模块功能介绍 该模块的使用介绍请参考dubbo官方用户手册如下章节内容. 集群容错 负载均衡 路由规则 配置规则 注册中心参考 ...

  9. 【转】Spark源码分析之-scheduler模块

    原文地址:http://jerryshao.me/architecture/2013/04/21/Spark%E6%BA%90%E7%A0%81%E5%88%86%E6%9E%90%E4%B9%8B- ...

最新文章

  1. SAP PP ECM的生效日期如何影响工单?
  2. Wannafly挑战赛29题解
  3. 趣学python编程第六章答案_Python核心编程-第六章-习题
  4. oracle alter_log,在线查看alter.log文件内容
  5. java 图片服务器 上传_Java实现把图片上传到图片服务器(nginx+vsftp)
  6. Ubuntu下使用VNC
  7. 修改Win7的默认文件夹排列方式
  8. web前端技术分享文档
  9. 苹果内部应用商店批准/审核 iPhone 应用程序相关流程
  10. 编译原理-回溯法及其优化的思考
  11. truncate 释放表空间
  12. 冲刺IPO:Soul在深红市场打开的社交元宇宙
  13. 双频段AP wps同时开启并通过智能手机、windows端连接问题
  14. 使用VS Code 配置 LaTeX 编辑器
  15. 什么是RPC?RPC好处?常用的RPC框架?
  16. 原生小程序 之引入 icon字体图标
  17. 滞胀世代 (ZT) 作者:bystander
  18. python画流星_用python一起来看流星雨
  19. Go语言学习资料大全整理
  20. 专访:平安科技首席架构师金新明和他的程序人生

热门文章

  1. 中心信令服务器编码,基于SIP的中心信令控制服务器①.PDF
  2. 化学专业计算机课程设计,分析化学专业课程设计.doc
  3. 动态规划习题,关系式推导和求模技巧
  4. php删除记录前的判断弹窗,thinkPHP删除前弹出确认框的简单实现方法
  5. centos7 mysql安装配置_mysql 5.7.17 安装配置方法图文教程(CentOS7)
  6. java paint方法哪个周期调用_关于一段java程序的小问题,paint函数是什么时候执行的,没有调用啊...
  7. java求导数_OO_JAVA_表达式求导
  8. window服务器cpu过高的排查_高频面试题:Java程序占用 CPU 过高怎么排查
  9. linux:进程之间的通信
  10. 包含目录、库目录、附加包含目录、附加库目录、附加依赖项之详解