ElasticSearch从入门到精通–第七话（自动补全、拼音分词器、自定义分词、数据同步方案）

使用拼音分词

可以引入elasticsearch的拼音分词插件，地址:https://github.com/medcl/elasticsearch-analysis-pinyin

下载后，将包上传至服务器后，解压缩

unzip -d py elasticsearch-analysis-pinyin-7.12.1.zip

将压缩后的目录放入es的plugins中即可，我这边是docker的数据卷，直接放入就行
```
cp -r py/ /var/lib/docker/volumes/es-plugins/_data
```
重启es服务
```
docker restart es
```

进入kibana控制台，使用拼音分词器看下结果

POST请求/analyze

{"analyzer": "pinyin","text": "学习es的很多天"
}

是按照拼音进行分词的。

自定义分词器

实现方式就是先走ik分词器，然后再走pinyin分词器进行拼音处理。

我们可以在创建索引库时，通过settings来配置自定义的analyzer(分词器)

实例：

PUT /test
{"settings": {"analysis": {"analyzer": { "my_analyzer": {  // 自定义 my_analyzer分词器"tokenizer": "ik_max_word", // 先经过ik处理"filter": "py"  // 在经过拼音处理}},"filter": {"py": { "type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"name":{"type": "text","analyzer": "my_analyzer"  // 使用自定义分词器}}}
}

测试

POST  /test/_analyze
{"analyzer": "my_analyzer","text": "学习es的很多天"
}

中英文的分词都具有了。

完整版索引创建+分词器

但是拼音分词器适合在创建倒排索引的时候使用，不能在搜索的时候使用。假如遇见同音不同字的，搜索时是按照中文搜索的，那么所有同音的就会全都被搜索出来，会出问题。！

如下：在文档中新增狮子、虱子后，查询时中文带狮子，但是因为查询时也同时使用了拼音分词器，所以虱子也会被查询

所以，我们可以在创建索引时使用ik和pinyin分词器，但是查询时还是要根据用户输入的是中文还是拼音去选择分词器。

DELETE /testPUT /test
{"settings": {"analysis": {"analyzer": { "my_analyzer": { "tokenizer": "ik_max_word","filter": "py"}},"filter": {"py": { "type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"name":{"type": "text","analyzer": "my_analyzer","search_analyzer": "ik_smart" // 查询时使用ik分词器}}}
}

自动补全

es提供了Completion Suggester查询来实现自动补全功能。这个查询会匹配用户输入内容开头的词条并且返回。条件是：参与补全查询的字段必须设置为completion类型，字段的内容一般是用来补全的多个词条形成的数组

查询时不再用query，而是使用的suggest

示例:

DELETE test
PUT test
{"mappings": {"properties": {"title":{"type": "completion"}}}
}// 插入数据
POST test/_doc
{"title": ["Sony", "WH-1000XM3"]
}
POST test/_doc
{"title": ["SK-II", "PITERA"]
}
POST test/_doc
{"title": ["Nintendo", "switch"]
}

查询

GET /test/_search
{"suggest": {"title_suggest": {  // 自定义查询名"prefix": "s", // 关键字,这里有写text的，也有写prefix的，效果类似"completion": {"field": "title", // 补全字段"skip_duplicates": true, // 跳过重复的"size": 10 // 获取前10条结果}}}
}

完整版demo

PUT /hotel
{"settings": {"analysis": {"analyzer": {"text_anlyzer": {"tokenizer": "ik_max_word","filter": "py"},"completion_analyzer": {"tokenizer": "keyword","filter": "py"}},"filter": {"py": {"type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"id":{"type": "keyword"},"name":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart","copy_to": "all"},"address":{"type": "keyword","index": false},"price":{"type": "integer"},"score":{"type": "integer"},"brand":{"type": "keyword","copy_to": "all"},"city":{"type": "keyword"},"starName":{"type": "keyword"},"business":{"type": "keyword","copy_to": "all"},"location":{"type": "geo_point"},"pic":{"type": "keyword","index": false},"all":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart"},"suggestion":{"type": "completion","analyzer": "completion_analyzer"}}}
}

RestAPI实现自动补全+结果处理

public class ESTEST_API_Query {public static void main(String[] args) throws IOException {// 创建es客户端,指定ip和端口RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("192.168.1.40", 9200, "http")));SearchRequest request = new SearchRequest("hotel");request.source().suggest(new SuggestBuilder().addSuggestion("mySuggestion",SuggestBuilders.completionSuggestion("title").prefix("h")    // 用户输入的关键字.skipDuplicates(true)   // 跳过重复.size(10)   // 取10条));SearchResponse response = client.search(request, RequestOptions.DEFAULT);client.close();}
}

结果处理

数据同步

es中很多数据都是来自mysql的，因为mysql数据发生改变时，es必须要跟着改变数据，就是es和mysql之间的数据同步.

实现简单，粗暴。但是业务耦合度较高。

低耦合，实现难度一般，需要依赖mq的可靠性