使用Springboot整合ElasticSearch以及通过接口热更新分词和同义词

什么是ElasticSearch

Elasticsearch(ES) 是一个基于Lucene构建的开源、分布式、RESTful接口全文搜索引擎。ElasticSearch还是一个分布式文档数据库，其中每个字段均被索引且可被搜索，它能够扩展至数以百计的服务器存储以及处理PB级的数据。他可以在很短的时间内存储、搜索和分析大量的数据。

demo地址 : https://github.com/BKTiger/elasticsearch-demo.git

java与ES整合配置

引入依赖
本项目使用的Springboot版本为2.0.7.RELEASE，注意必须指定elasticsearch版本，否则依赖会报错。

<properties><project.build.sourceEncoding>UTF-8</project.build.sourceEncoding><project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding><java.version>1.8</java.version><elasticsearch.version>7.1.0</elasticsearch.version> <!-- 这里须指定ElasticSearch版本-->
</properties><dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>${elasticsearch.version}</version>
</dependency>

引入配置
2.1 简版配置

@Configuration
public class ElasticSearchRestConfig {@Beanpublic RestHighLevelClient restHighLevelClient(){RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http")));return client;}
}

2.2 终板配置

@Configuration
public class ElasticSearchRestConfig {@Value("${data.elasticsearch.url}")private String[] ipAddress;private static final int ADDRESS_LENGTH = 2;private static final String HTTP_SCHEME = "http";@Beanpublic RestClientBuilder restClientBuilder() {HttpHost[] hosts = Arrays.stream(ipAddress).map(this::makeHttpHost).filter(Objects::nonNull).toArray(HttpHost[]::new);return RestClient.builder(hosts);}@Bean@Primarypublic RestHighLevelClient highLevelClient(@Autowired RestClientBuilder restClientBuilder) {//TODO 此处可以进行其它操作return new RestHighLevelClient(restClientBuilder);}private HttpHost makeHttpHost(String s) {String[] address = s.split(":");if (address.length == ADDRESS_LENGTH) {String ip = address[0];int port = Integer.parseInt(address[1]);return new HttpHost(ip, port, HTTP_SCHEME);} else {return null;}}
}

索引的构建

索引的创建
创建代码参考demo的test方法中的 com.zhangtai.demo.DemoBootApplicationTests#createIndex，该方法中引入了ik分词器和dynamic-synonym同义词插件，下文中有使用和配置的方法，需要先加入ik分词器和dynamic-synonym同义词插件后才可使用该方法创建索引，如果不使用可以不进行索引的人工创建，es会自行创建，分词器使用默认分词器，每个汉字为一个分词。
索引的删除

 // DELETE http://127.0.0.1:9200/goodsprivate void deleteAll(String index){DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(index);try {restHighLevelClient.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}}

判断索引是否存在

 public void indexExist(){// goods索引是否存在GetIndexRequest goods = new GetIndexRequest("goods");try {boolean exists = restHighLevelClient.indices().exists(goods, RequestOptions.DEFAULT);System.out.println("goods索引存在与否:"+exists);} catch (IOException e) {e.printStackTrace();}}

数据操作

新增数据
1.1 实体类

@Data
public class Goods {private String id;private String name;private BigDecimal price;private Long skuId;private Long spuId;private Long colorId;private String colorName;public Goods() {}public Goods(String id, String name, BigDecimal price, Long skuId, Long spuId, Long colorId, String colorName) {this.id = id;this.name = name;this.price = price;this.skuId = skuId;this.spuId = spuId;this.colorId = colorId;this.colorName = colorName;}public Goods(String name, BigDecimal price, Long skuId, Long spuId, Long colorId, String colorName) {this.name = name;this.price = price;this.skuId = skuId;this.spuId = spuId;this.colorId = colorId;this.colorName = colorName;}
}

2.2 新增测试数据

 private List<Map<String,Object>> getDate(){List<Map<String,Object>> result = new ArrayList<>();Goods goods = new Goods("00001","袜子",new BigDecimal(21.3D),1L,1L,1L,"红色");String s = JSON.toJSONString(goods);Map map = JSON.parseObject(s, Map.class);result.add(map);goods = new Goods("00002","袜子",new BigDecimal(21.3D),2L,1L,2L,"白色");s = JSON.toJSONString(goods);map = JSON.parseObject(s, Map.class);result.add(map);goods = new Goods("00003","袜子",new BigDecimal(21.3D),3L,1L,3L,"黑色");s = JSON.toJSONString(goods);map = JSON.parseObject(s, Map.class);result.add(map);goods = new Goods("00004","裤子牛仔裤直筒休闲",new BigDecimal(50.3D),4L,2L,1L,"红色");s = JSON.toJSONString(goods);map = JSON.parseObject(s, Map.class);result.add(map);goods = new Goods("00005","裤子牛仔裤直筒休闲",new BigDecimal(50.3D),5L,2L,2L,"白色");s = JSON.toJSONString(goods);map = JSON.parseObject(s, Map.class);result.add(map);return result;}

2.3 保存数据

   /* POST http://127.0.0.1:9200/goods/_doc/00005{"name":"裤子","skuId":"5","spuId":"2","colorId":"3","price":90.3,"colorName":"绿色"} */public void save() {// 获取数据List<Map<String, Object>> date = getDate();// 单个插入
//      date.forEach(data -> {//          saveDate("goods",data);
//      });// 批量插入saveDateAll("goods",date);}/*** 功能描述: 保存单条数据** @param: [index, data]* @return: void* @auther: zhangtai* @date: 2020/1/8 18:00*/
private void saveDate(String index,Map<String, Object> data){IndexRequest indexRequest = new IndexRequest(index).source(data).id(data.get("id").toString());try {System.out.println(indexRequest.toString());restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}
}/*** 功能描述: 批量保存数据** @param: [index, datas]* @return: void* @auther: zhangtai* @date: 2020/1/8 18:00*/
private void saveDateAll(String index,List<Map<String, Object>> datas){if(!CollectionUtils.isEmpty(datas)){BulkRequest bulkRequest = new BulkRequest();datas.forEach(data->{IndexRequest indexRequest = new IndexRequest(index).source(data).id(data.get("id").toString());bulkRequest.add(indexRequest);});try {System.out.println(bulkRequest.toString());restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}}}

更新数据

 /* POST http://127.0.0.1:9200/goods/_doc/00003/_update{"doc":{"price":12.4}} */public void update(){Map<String,Object> param = new HashMap<>();param.put("name","裤子");UpdateRequest goods = new UpdateRequest("goods", "00001").doc(param);try {restHighLevelClient.update(goods,RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}}

删除数据

// DELETE http://127.0.0.1:9200/goods/_doc/00002
public void delete(){DeleteRequest goods = new DeleteRequest("goods","00001");try {restHighLevelClient.delete(goods,RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}
}

根据id查询数据

// GET http://127.0.0.1:9200/goods/_doc/00003
public void getById(){GetRequest goods = new GetRequest("goods", "00003");GetResponse documentFields = null;try {documentFields = restHighLevelClient.get(goods, RequestOptions.DEFAULT);} catch (IOException e) {e.printStackTrace();}String sourceAsString = documentFields.getSourceAsString();Goods goods1 = JSON.parseObject(sourceAsString, Goods.class);System.out.println(goods1.toString());}

搜索

分页查询

 /* GET http://127.0.0.1:9200/goods/_doc/_search{"size":2,"from":1} */public void searchPage(){SearchRequest goods = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.size(2).from(0);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}}

排序

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"sort": [{"skuId": {"order": "desc"}}]
} */
public void searchSort(){SearchRequest goods = new SearchRequest("goods");//searchSourceBuilder.sort(SortBuilders.fieldSort("skuId").order(SortOrder.DESC));//searchSourceBuilder.sort(SortBuilders.fieldSort("skuId").order(SortOrder.ASC));//searchSourceBuilder.sort("skuId",SortOrder.DESC);MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name", "裤子");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(matchQueryBuilder);searchSourceBuilder.sort(SortBuilders.scoreSort().order(SortOrder.DESC));SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

term 相等的查询

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"term": {"name.keyword": {"value": "袜子"}}}
} */
public void searchTerm(){SearchRequest goods = new SearchRequest("goods");// TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name.keyword", "袜子");TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("colorId", 1L);//TermQueryBuilder termQueryBuilder = new TermQueryBuilder("colorId", 1L);SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(termQueryBuilder);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

terms in查询

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query":{"terms": {"skuId": [1,2]}}
} */
public void searchTerms(){SearchRequest goods = new SearchRequest("goods");TermsQueryBuilder termsQueryBuilder = QueryBuilders.termsQuery("skuId", new long[]{1L,2L});SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(termsQueryBuilder);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

wildcard 模糊匹配(like)

text会对字段进行分词处理而keyword则不会

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"wildcard": {"name.keyword": {"wildcard": "*子牛仔*","boost": 1}}}
} */
public void searchWildcard(){SearchRequest goods = new SearchRequest("goods");WildcardQueryBuilder name = QueryBuilders.wildcardQuery("name.keyword", "*子牛仔*");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(name);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

range 范围查询

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"range": {"skuId": {"from": null,"to": 2,"include_lower": true,"include_upper": false,"boost": 1}}}
} */
public void searchRange(){SearchRequest goods = new SearchRequest("goods");// 范围//RangeQueryBuilder skuIds = QueryBuilders.rangeQuery("skuId").gte(1L).lte(1L);// >=//RangeQueryBuilder skuIds = QueryBuilders.rangeQuery("skuId").gte(1L);// >//RangeQueryBuilder skuIds = QueryBuilders.rangeQuery("skuId").gt(1L);// <=//RangeQueryBuilder skuIds = QueryBuilders.rangeQuery("skuId").lte(2L);// <RangeQueryBuilder skuIds = QueryBuilders.rangeQuery("skuId").lt(2L);SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(skuIds);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

match 分词查询

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query":{"match":{"name":"裤子"}}
} */
public void searchMatch(){SearchRequest goods = new SearchRequest("goods");//MatchQueryBuilder name = QueryBuilders.matchQuery("name", "直筒");MatchQueryBuilder name = QueryBuilders.matchQuery("name", "裤子直筒");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(name);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

multi_match 分词查询(匹配多词)

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"multi_match": {"query": "红色 裤子","fields": ["colorName^1.0","name^1.0"],"boost": 1}},"explain":true //结果中会对匹配结果进行说明
}*/
public void searchMatchAll(){SearchRequest goods = new SearchRequest("goods");MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder("裤子红色","name","colorName");
//      增加权重
//      Map<String,Float> fields = new HashMap<>();
//      fields.put("name",1.0F);
//      fields.put("colorName",2.0F);
//      MultiMatchQueryBuilder multiMatchQueryBuilder = new MultiMatchQueryBuilder("裤子红色").fields();SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(multiMatchQueryBuilder);System.out.println(searchSourceBuilder);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

bool must条件为and，should条件为or条件

GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"bool": {"must": [{"term": {"colorId": {"value": 1,"boost": 1}}},{"wildcard": {"name.keyword": {"wildcard": "*袜子*","boost": 1}}}],"adjust_pure_negative": true,"boost": 1}}
}
public void searchBoolMust(){SearchRequest goods = new SearchRequest("goods");TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("colorId", 1L);WildcardQueryBuilder name = QueryBuilders.wildcardQuery("name.keyword", "*裤子*");BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();boolQueryBuilder.must(termQueryBuilder).must(name);SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(boolQueryBuilder);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

查询和过滤

使用过滤的时候,es不会对条件进行打分,效率会比查询快

public void searchFilter(){SearchRequest goods = new SearchRequest("goods");BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();MatchQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("name", "裤子");boolQueryBuilder.filter(matchQueryBuilder);SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.query(boolQueryBuilder);//searchSourceBuilder.query(matchQueryBuilder);searchSourceBuilder.explain(true);searchSourceBuilder.sort(SortBuilders.scoreSort());SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

查询指定字段

public void searchIncluds(){SearchRequest goods = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();searchSourceBuilder.fetchSource(new String[]{"name","skuId"},null);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

折叠

/* GET http://127.0.0.1:9200/goods/_doc/_search
{"collapse": {"field": "spuId"}
} */
public void searchCollapse(){SearchRequest goods = new SearchRequest("goods");SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();CollapseBuilder spuId = new CollapseBuilder("spuId");searchSourceBuilder.collapse(spuId);SearchRequest source = goods.source(searchSourceBuilder);try {SearchResponse search = restHighLevelClient.search(source, RequestOptions.DEFAULT);long total = search.getHits().getTotalHits().value;System.out.println(total);SearchHit[] hits = search.getHits().getHits();for (SearchHit hit : hits) {String sourceAsString = hit.getSourceAsString();System.out.println(sourceAsString);}} catch (IOException e) {e.printStackTrace();}
}

折叠后总数统计不准确

权重

GET http://127.0.0.1:9200/goods/_doc/_search
{"query": {"bool": {"should": [{"term": {"skuId": {"value": 5,"boost": 12}}},{"term": {"colorId": {"value": 1,"boost": 6}}}],"boost": 1}},"sort": [{"_score": {"order": "desc"}}],"explain":true
}

高级

分片和备份

cluster：代表一个集群，集群中有多个节点，其中有一个为主节点，这个主节点是可以通过选举产生的，主从节点是对于集群内部来说的。es的一个概念就是去中心化，字面上理解就是无中心节点，这是对于集群外部来说的，因为从外部来看es集群，在逻辑上是个整体，你与任何一个节点的通信和与整个es集群通信是等价的。

shards：代表索引分片，es可以把一个完整的索引分成多个分片，这样的好处是可以把一个大的索引拆分成多个，分布到不同的节点上。构成分布式搜索。分片的数量只能在索引创建前指定，并且索引创建后不能更改。

replicas：代表索引副本，es可以设置多个索引的副本，副本的作用一是提高系统的容错性，当某个节点某个分片损坏或丢失时可以从副本中恢复。二是提高es的查询效率，es会自动对搜索请求进行负载均衡。
1. 默认1个分片,一个备份

加入ik分词器

ik分词器插件地址 : https://github.com/medcl/elasticsearch-analysis-ik

使用方法

 - 下载项目- mvn package- 拷贝elasticsearch-analysis-ik-7.1.0\target\releases\elasticsearch-analysis-ik-7.0.0.zip 到目录 elasticsearch-7.1.0\plugins- 重启ES

加入同义词

同义词插件地址 : https://github.com/bells/elasticsearch-analysis-dynamic-synonym 使用方式参照本人另外一个博客
重复数据的处理
1. 对数据给出明确的排序
2. 传入查询时间，只查询指定时间前的数据
3. 使用scroll

使用Springboot整合ElasticSearch以及通过接口热更新分词和同义词相关推荐

es springboot 不设置id_原创 | 一篇解决Springboot 整合 Elasticsearch
ElasticSearch 结合业务的场景,在目前的商品体系需要构建搜索服务,主要是为了提供用户更丰富的检索场景以及高速,实时及性能稳定的搜索服务. ElasticSearch是一个基于Lucene的 ...
SpringBoot整合ElasticSearch实现多版本的兼容
前言在上一篇学习SpringBoot中,整合了Mybatis.Druid和PageHelper并实现了多数据源的操作.本篇主要是介绍和使用目前最火的搜索引擎ElastiSearch,并和Spring ...
SpringBoot 整合ElasticSearch全文检索
ElasticSearch是一个基于Lucene的搜索服务器.它提供了一个分布式多用户能力的全文搜索引擎,基于RESTful web接口.Elasticsearch是用Java语言开发的,并作为Apa ...
Elasticsearch的安装，以及Springboot整合Elasticsearch
*一.下载好elasticsearch并解压我这里用的是elasticsearch-5.6.8,下面是下载地址 https://artifacts.elastic.co/downloads/elas ...
SpringBoot整合Elasticsearch详细步骤以及代码示例（附源码）
准备工作# 环境准备# JAVA版本 Copy java version "1.8.0_121" Java(TM) SE Runtime Environment (build 1. ...
【十九】springboot整合ElasticSearch实战（万字篇）
本章开始学习springboot整合ElasticSearch 7.X版本并通过小demo实现基本的增删改查.实现如下案例: 1.当向数据新增一个商品信息时,同时向rabbitMQ发起消息(异步实现) ...
Springboot 整合ElasticSearch 常用的插入查询，模糊查询，范围查询
前言本来该篇教程就应该写到 Springboot 整合 ElasticSearch 入门教学必看 https://blog.csdn.net/qq_35387940/article/details/ ...
Springboot 整合 ElasticSearch 入门教学必看
ElasticSearch 相比搜到这篇文章的人,都已经有过对它的了解, 一种流行的企业级搜索引擎,是一个分布式,高性能.高可用.可伸缩的搜索和分析系统. 那么用我粗俗的言语来说,它就是提供一个存储数 ...
【数据篇】SpringBoot 整合 Elasticsearch 实践数据搜索引擎
写在最前 Elasticsearch 入门必读 Docker安装ELK Spring Data Elasticsearch 参考文档版本选择 Spring Data Release Train Sp ...