elasticsearch index 之 put mapping

mapping机制使得elasticsearch索引数据变的更加灵活,近乎于no schema。mapping可以在建立索引时设置,也可以在后期设置。后期设置可以是修改mapping(无法对已有的field属性进行修改,一般来说只是增加新的field)或者对没有mapping的索引设置mapping。put mapping操作必须是master节点来完成,因为它涉及到集群matedata的修改,同时它跟index和type密切相关。修改只是针对特定index的特定type。

在Action support分析中我们分析过几种Action的抽象类型,put mapping Action属于TransportMasterNodeOperationAction的子类。它实现了masterOperation方法,每个继承自TransportMasterNodeOperationAction的子类都会根据自己的具体功能来实现这个方法。这里的实现如下所示:

    protected void masterOperation(final PutMappingRequest request, final ClusterState state, final ActionListener<PutMappingResponse> listener) throws ElasticsearchException {final String[] concreteIndices = clusterService.state().metaData().concreteIndices(request.indicesOptions(), request.indices());      //构造requestPutMappingClusterStateUpdateRequest updateRequest = new PutMappingClusterStateUpdateRequest().ackTimeout(request.timeout()).masterNodeTimeout(request.masterNodeTimeout()).indices(concreteIndices).type(request.type()).source(request.source()).ignoreConflicts(request.ignoreConflicts());//调用putMapping方法,同时传入一个ListenermetaDataMappingService.putMapping(updateRequest, new ActionListener<ClusterStateUpdateResponse>() {@Overridepublic void onResponse(ClusterStateUpdateResponse response) {listener.onResponse(new PutMappingResponse(response.isAcknowledged()));}@Overridepublic void onFailure(Throwable t) {logger.debug("failed to put mappings on indices [{}], type [{}]", t, concreteIndices, request.type());listener.onFailure(t);}});}

以上是TransportPutMappingAction对masterOperation方法的实现,这里并没有多少复杂的逻辑和操作。具体操作在matedataMappingService中。跟之前的CreateIndex一样,put Mapping也是向master提交一个updateTask。所有逻辑也都在execute方法中。这个task的基本跟CreateIndex一样,也需要在给定的时间内响应。它的代码如下所示:

 public void putMapping(final PutMappingClusterStateUpdateRequest request, final ActionListener<ClusterStateUpdateResponse> listener) {//提交一个高基本的updateTaskclusterService.submitStateUpdateTask("put-mapping [" + request.type() + "]", Priority.HIGH, new AckedClusterStateUpdateTask<ClusterStateUpdateResponse>(request, listener) {@Overrideprotected ClusterStateUpdateResponse newResponse(boolean acknowledged) {return new ClusterStateUpdateResponse(acknowledged);}@Overridepublic ClusterState execute(final ClusterState currentState) throws Exception {List<String> indicesToClose = Lists.newArrayList();try {            //必须针对已经在matadata中存在的index,否则抛出异常for (String index : request.indices()) {if (!currentState.metaData().hasIndex(index)) {throw new IndexMissingException(new Index(index));}}//还需要存在于indices中,否则无法进行操作。所以这里要进行预建for (String index : request.indices()) {if (indicesService.hasIndex(index)) {continue;}final IndexMetaData indexMetaData = currentState.metaData().index(index);              //不存在就进行创建IndexService indexService = indicesService.createIndex(indexMetaData.index(), indexMetaData.settings(), clusterService.localNode().id());indicesToClose.add(indexMetaData.index());// make sure to add custom default mapping if existsif (indexMetaData.mappings().containsKey(MapperService.DEFAULT_MAPPING)) {indexService.mapperService().merge(MapperService.DEFAULT_MAPPING, indexMetaData.mappings().get(MapperService.DEFAULT_MAPPING).source(), false);}// only add the current relevant mapping (if exists)if (indexMetaData.mappings().containsKey(request.type())) {indexService.mapperService().merge(request.type(), indexMetaData.mappings().get(request.type()).source(), false);}}//合并更新MappingMap<String, DocumentMapper> newMappers = newHashMap();Map<String, DocumentMapper> existingMappers = newHashMap();            //针对每个index进行Mapping合并for (String index : request.indices()) {IndexService indexService = indicesService.indexServiceSafe(index);// try and parse it (no need to add it here) so we can bail early in case of parsing exceptionDocumentMapper newMapper;DocumentMapper existingMapper = indexService.mapperService().documentMapper(request.type());if (MapperService.DEFAULT_MAPPING.equals(request.type())) {//存在defaultmapping则合并default mapping// _default_ types do not go through merging, but we do test the new settings. Also don't apply the old defaultnewMapper = indexService.mapperService().parse(request.type(), new CompressedString(request.source()), false);} else {newMapper = indexService.mapperService().parse(request.type(), new CompressedString(request.source()), existingMapper == null);if (existingMapper != null) {// first, simulateDocumentMapper.MergeResult mergeResult = existingMapper.merge(newMapper, mergeFlags().simulate(true));// if we have conflicts, and we are not supposed to ignore them, throw an exceptionif (!request.ignoreConflicts() && mergeResult.hasConflicts()) {throw new MergeMappingException(mergeResult.conflicts());}}}newMappers.put(index, newMapper);if (existingMapper != null) {existingMappers.put(index, existingMapper);}}String mappingType = request.type();if (mappingType == null) {mappingType = newMappers.values().iterator().next().type();} else if (!mappingType.equals(newMappers.values().iterator().next().type())) {throw new InvalidTypeNameException("Type name provided does not match type name within mapping definition");}if (!MapperService.DEFAULT_MAPPING.equals(mappingType) && !PercolatorService.TYPE_NAME.equals(mappingType) && mappingType.charAt(0) == '_') {throw new InvalidTypeNameException("Document mapping type name can't start with '_'");}final Map<String, MappingMetaData> mappings = newHashMap();for (Map.Entry<String, DocumentMapper> entry : newMappers.entrySet()) {String index = entry.getKey();// do the actual merge here on the master, and update the mapping sourceDocumentMapper newMapper = entry.getValue();IndexService indexService = indicesService.indexService(index);if (indexService == null) {continue;}CompressedString existingSource = null;if (existingMappers.containsKey(entry.getKey())) {existingSource = existingMappers.get(entry.getKey()).mappingSource();}DocumentMapper mergedMapper = indexService.mapperService().merge(newMapper.type(), newMapper.mappingSource(), false);CompressedString updatedSource = mergedMapper.mappingSource();if (existingSource != null) {if (existingSource.equals(updatedSource)) {// same source, no changes, ignore it} else {// use the merged mapping sourcemappings.put(index, new MappingMetaData(mergedMapper));if (logger.isDebugEnabled()) {logger.debug("[{}] update_mapping [{}] with source [{}]", index, mergedMapper.type(), updatedSource);} else if (logger.isInfoEnabled()) {logger.info("[{}] update_mapping [{}]", index, mergedMapper.type());}}} else {mappings.put(index, new MappingMetaData(mergedMapper));if (logger.isDebugEnabled()) {logger.debug("[{}] create_mapping [{}] with source [{}]", index, newMapper.type(), updatedSource);} else if (logger.isInfoEnabled()) {logger.info("[{}] create_mapping [{}]", index, newMapper.type());}}}if (mappings.isEmpty()) {// no changes, returnreturn currentState;}//根据mapping的更新情况重新生成matadataMetaData.Builder builder = MetaData.builder(currentState.metaData());for (String indexName : request.indices()) {IndexMetaData indexMetaData = currentState.metaData().index(indexName);if (indexMetaData == null) {throw new IndexMissingException(new Index(indexName));}MappingMetaData mappingMd = mappings.get(indexName);if (mappingMd != null) {builder.put(IndexMetaData.builder(indexMetaData).putMapping(mappingMd));}}return ClusterState.builder(currentState).metaData(builder).build();} finally {for (String index : indicesToClose) {indicesService.removeIndex(index, "created for mapping processing");}}}});}

以上就是mapping的设置过程,首先它跟Create index一样,只有master节点才能操作,而且是以task的形式提交给master;其次它的本质是将request中的mapping和index现存的或者是default mapping合并,并最终生成新的matadata更新到集群的各个节点。

总结:集群中的master操作无论是index方面还是集群方面,最终都是集群matadata的更新过程。而这些操作只能在master上进行,并且都是会超时的任务。put mapping当然也不例外。上面的两段代码基本概况了mapping的设置过程。这里就不再重复了。这里还有一个问题没有涉及到就是mapping的合并。mapping合并会在很多地方用到。

转自:http://www.cnblogs.com/zziawanblog/p/7011367.html

转载于:https://www.cnblogs.com/bonelee/p/7382326.html

elasticsearch index 之 put mapping相关推荐

  1. ElasticSearch index 剖析

    ElasticSearch index 剖析 在看ElasticSearch权威指南基础入门中关于:分片内部原理这一小节内容后,大致对ElasticSearch的索引.搜索底层实现有了一个初步的认识. ...

  2. Go Elasticsearch index CRUD

    文章目录 1.简介 2.增加 2.删除 3.修改 3.1 更新 mapping 3.1.1 增加字段 3.1.2 删除字段 3.1.3 添加 multi-fields 3.2 重命名 index 4. ...

  3. 解决elasticsearch里拒绝更新mapping设置的错误

    @[TOC] 解决elasticsearch里拒绝更新mapping设置的错误 错误现象 对一个es上已经创建的索引,使用python的elasticsearch_dsl 库, Rejecting m ...

  4. elasticsearch index、create和update的源码分析

    https://segmentfault.com/a/1190000011272749 社区里面有人问了如下一个问题: 执行 bulk 索引文档的时候,用 index 或者 create 类型并且自定 ...

  5. 【Elasticsearch】es Root mapping definition has unsupported parameters

    1.概述 转载:https://blog.csdn.net/u014646662/article/details/94718834 ElasticSearch 7.x 默认不在支持指定索引类型,在el ...

  6. ElasticSearch Index Settings

    目录 Index Settings 静态索引配置 index.number_of_shards index.shard.check_on_startup index.codec index.routi ...

  7. [Elasticsearch] Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters

    一.ES7报错 Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters 原因:es7不建议 ...

  8. 【Elasticsearch教程18】Mapping字段类型之text 以及term、match和analyzer

    Elasticsearch Mapping字段类型之text 以及term.match和analyzer 一.text场景 二.`term`查询 三.`match`查询 1. `亚瑟王`如何存储? 2 ...

  9. ElasticSearch系列18:Mapping 设计指南

     点击上方"方才编程",即可关注我! 本文导读 ElasticSearch 的 mapping 该如何设计,才能保证检索的高效?想要回答这个问题,就需要全面系统地掌握 mappin ...

最新文章

  1. 一个小问题引发的论证思考
  2. 智慧政务解决方案(28页)pdf_智慧政务解决方案在政务服务大厅中的应用
  3. vrep中设置joint的位置、速度需要根据关节的模式来设置。
  4. linux下用户态程序coredump生成方法
  5. obs多推流地址_基于腾讯云的OBS 推流
  6. c++:std::dec, std::hex, std::oct
  7. 佳能打印机手机显示未连接服务器,佳能打印机出现服务器设置密码
  8. node.js(四 --- 全局对象)
  9. 指尖江湖鸿蒙抽奖,剑网三:指尖江湖 李忘生竞技场攻略
  10. 程序员非常好用的app
  11. Httpd服务重定向配置
  12. USB设备无法识别也无法读取怎么办?
  13. 第十一节:分布式文件系统
  14. 【DBA100人】李建明:一名普通DBA的14年技术之路与成长智慧
  15. 亚马逊账户锁定无法登陆_如何删除您的亚马逊账户
  16. .net core 使用 ZKWeb.system.drawing 在centos下使用gdi 画图
  17. Threadx tx_thread_create创建线程
  18. 1.65亿融资背后,是时候把「百度」标签从林元庆身上摘下了| 人物特写
  19. CSS flex的一些属性
  20. linux查看工作流,工作流开发图文教程(WorkFlow Developer) 之 二 WorkFlow调试

热门文章

  1. 高级mysql优化知识_MySQL高级第三篇(索引优化分析)
  2. mac java jdk_mac下java JDK的下载安装和配置
  3. 关闭 定时开启_【话说定时器系列】之四:STM32定时器更新事件及案例分享
  4. python 画云图_【词云图】如何用python的第三方库jieba和wordcloud画词云图
  5. mysql的in查询是可以用到索引吗?亲测详解
  6. 如何保证高可用?java测试工程师测试的方法
  7. java读取文件替换字符,跳槽薪资翻倍
  8. 【深度学习】实战Kaggle竞赛之线性模型解决波士顿房价预测问题(Pytorch)
  9. 【408预推免复习】计算机组成原理之计算机的发展及应用
  10. 五十音图平假名流氓记忆(MD~!真难)