ES

1、ElasticSearch概述

2、ES与Solr的差别

①Solr简介

②Lucene简介

③ES VS Solr

3、ElasticSearch 安装

https://www.elastic.co/cn/

进入bin目录，打开.bat文件，这里用windows版本演示

启动在9200端口上

测试访问

es head 可视化插件

使用之前请先配置/依赖nodeJS环境

#下载链接
wget  https://github.com/mobz/elasticsearch-head/archive/master.zip
cd elasticsearch-head-master
npm install
npm run start

这里涉及到跨域的问题，需要配置，关闭es，打开es配置文件配置跨域，然后在重启es

http.cors.enabled: true
http.cors.allow-origin: "*"

连接成功！！！

了解ELK

4、Kibana安装

开箱即用，注意需要跟es版本对应！！！，解压的时间会比较长

https://www.elastic.co/cn/downloads/kibana

启动

访问localhost:5601

汉化

\config\kibana.yml

server.port: 5601
server.host: "0.0.0.0"
elasticsearch.hosts: ["http://192.168.1.30:9201"]
kibana.index: ".kibana"
i18n.locale: "zh-CN"            # 中文汉化

汉化成功！！！

5、ES核心概念

索引
字段类型（mapping）
文档（document）

一切都是JSON

6、IK分词器

下载链接
解压放入到es对应的plugins下即可

重启观察ES，发现ik插件被加载了

也可以通过bin目录下elasticsearch-plugin list 查看已经加载的插件
使用kibana测试
- ik_smart: 最少切分
- ik_max_word为最细粒度划分！
  
  穷尽词库的可能，字典！
ik分词器增加自己的配置与字典

plugins\ik\config下

重启ES 和 Kibana

7、 Restful风格说明

基础测试

创建一个索引

PUT /索引名/~类型名~/文档id
{请求体}

# 指令
put /test/achang/1
{"name":"achang","age":18,"address":"中国"
}# 结果
{"_index" : "test",# 索引"_type" : "achang",# 类型（已经废弃）"_id" : "1",# id"_version" : 1,# 版本"result" : "created",# 操作类型"_shards" : {# 分片信息"total" : 2,"successful" : 1,"failed" : 0},"_seq_no" : 0,"_primary_term" : 1
}

es字段类型

设置索引库的参数类型，创建规则

获取具体的索引规则

# GET test2{"test2" : {"aliases" : { },"mappings" : {"properties" : {"age" : {"type" : "integer"},"birthday" : {"type" : "date"},"name" : {"type" : "text"}}},"settings" : {"index" : {"creation_date" : "1599708623941","number_of_shards" : "1","number_of_replicas" : "1","uuid" : "ANWnhwArSMSl8k8iipgH1Q","version" : {"created" : "7080099"},"provided_name" : "test2"}}}
}# 查看默认的规则
PUT /test3/_doc/1
{"name": "狂神说Java","age": 28,"birthday": "1997-01-05"
}# GET test3{"test3" : {"aliases" : { },"mappings" : {"properties" : {"age" : {"type" : "long"},"birthday" : {"type" : "date"},"name" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}},"settings" : {"index" : {"creation_date" : "1599708906181","number_of_shards" : "1","number_of_replicas" : "1","uuid" : "LzPLCDgeQn6tdKo3xBBpbw","version" : {"created" : "7080099"},"provided_name" : "test3"}}}
}

修改索引 POST

# 只会修改指定项，其他内容保证不变
#post /test/achang/1
{"name":"achang!!!","age":20,"address":"中国"
}#get /test/_doc/1
{"_index" : "test","_type" : "achang","_id" : "1","_version" : 2,#版本号加1，乐观锁"_seq_no" : 1,"_primary_term" : 1,"found" : true,"_source" : {"name" : "achang!!!","age" : 20,"address" : "中国"}
}

删除索引

8、关于文档的基本操作

基本操作（简单的查询）

put /achang/user/1
{"name": "阿昌","age": 22,"desc": "一顿操作猛如虎，一看工资2500","tags": ["码农", "技术宅", "直男"]
}put /achang/user/2
{"name": "张三","age": 28,"desc": "法外狂徒","tags": ["旅游", "渣男", "交友"]
}put /achang/user/3
{"name": "李四1","age": 30,"desc": "不知道怎么描述","tags": ["旅游", "靓女", "唱歌"]
}GET achang/user/1GET achang/user/_search?q=name:阿昌

复杂操作(排序、分页、高亮、模糊查询、标准查询！)

# 模糊查询
GET achang/user/_search
{"query": {"match": {#模糊匹配"name": "阿"}}
}# 对查询结果进行字段过滤
GET achang/user/_search
{"query": {"match": {"name": "阿昌"}},"_source": ["name", "desc"]#代表你只要name、desc的值
}# 排序
GET achang/user/_search
{"query": {"match": {"name": "阿昌"}},"sort":[{"age": "asc"# 指定匹配规则desc、asc}]
}# 分页
GET achang/user/_search
{"query": {"match": {"name": "阿昌" #模糊匹配条件}},"sort":[{"age": "asc" #排序规则}], "from": 0,#从第几个开始"size": 2 #显示数量
}

布尔值条件查询

# 多条件查询 must 相当于and
GET achang/user/_search
{"query": {"bool": {"must": [#条件数组，模糊匹配{"match": {"name": "阿昌" #条件1}},{"match": {"age": 22 #条件2}}]}}
}# 多条件查询 should 相当于or
GET achang/user/_search
{"query": {"bool": {"should": [# or关系{"match": {"name": "阿昌" #条件1}},{"match": {"age": 25 #条件2}}]}}
}# 多条件查询 must_not 相当于 not
GET achang/user/_search
{"query": {"bool": {"must_not": [ # not关系{"match": {"age": 25 #条件，age不等于25的数据}}]}}
}# 过滤查询1 age > 24
GET achang/user/_search
{"query": {"bool": {"must": [ # 模糊匹配{"match": {"name": "1" #name含有1的}}],"filter": [ # 过滤条件{"range": { #范围"age": { #age大于24的"gt": 24}}}]}}
}# 过滤器2  22<age<30
GET achang/user/_search
{"query": {"bool": {"must": [{"match": {"name": "阿昌"}}],"filter": [ #过滤规则{"range": { #范围"age": { #年龄"lt": 30, #小于30"gt": 21 #大于21}}}]}}
}

多条件查询

多条件用空格分开，数据只要满足一个条件就会被查出

GET achang/user/_search
{"query": { #模糊匹配"match": { #匹配规则"tags": "技术 男" #tags含有技术、男}}
}

精准查询

keyword类型不会被分词器解析

term: 精确匹配倒排索引

# 定义类型
PUT xiaoachang_test_db
{"mappings": { #映射"properties": { #字段属性"name": {"type": "keyword"},"desc": {"type": "text"}}}
}PUT /xiaoachang_test_db/_doc/1
{"name": "阿昌Java Name","desc": "阿昌 Desc"
}PUT /xiaoachang_test_db/_doc/2
{"name": "阿昌Java Name","desc": "阿昌Java Desc 2"
}# 按照keyword类型精准匹配
GET xiaoachang_test_db/_search
{"query": {"term": { #倒排索引查询"name": "阿昌Java Name"}}
}
# 结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 0.6931471,"hits" : [{"_index" : "test_db","_type" : "_doc","_id" : "1","_score" : 0.6931471,"_source" : {"name" : "阿昌Java Name","desc" : "阿昌Java Desc"}}]}
}# 按照text类型匹配
GET xiaoachang_test_db/_search
{"query": {"term": { #倒排索引查询"name": "小"}}
}# 结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 2,"relation" : "eq"},"max_score" : 0.18232156,"hits" : [{"_index" : "test_db","_type" : "_doc","_id" : "1","_score" : 0.18232156,"_source" : {"name" : "阿昌Java Name","desc" : "阿昌Java Desc"}},{"_index" : "test_db","_type" : "_doc","_id" : "2","_score" : 0.18232156,"_source" : {"name" : "阿昌Java Name","desc" : "阿昌Java Desc 2"}}]}
}

多个值匹配精确查询

PUT /test_db/_doc/3
{"t1": "22","t2": "2020-09-10"
}PUT /test_db/_doc/4
{"t1": "33","t2": "2020-09-11"
}GET test_db/_search
{"query": {"bool": { #布尔判断"should": [ #or关系{"term": {"t1": "22" #精确判断，t1等于22}},{"term": {"t1": "33" #或t1等于33}}]}}
}

高亮查询

GET achang/user/_search
{"query": {"match": { #模糊查询"name": "阿昌"}},"highlight": { #高亮查询"pre_tags": "<p class='key' style='color:red'>", #前缀，左侧的key为css样式"post_tags": "</p>", #后缀"fields": { #设置需要高亮的字段"name": {}}}
}# 结果显示：
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 1,"relation" : "eq"},"max_score" : 2.0834165,"hits" : [{"_index" : "achang","_type" : "user","_id" : "1","_score" : 2.0834165,"_source" : {"name" : "阿昌","age" : 22,"desc" : "一顿操作猛如虎，一看工资2500","tags" : ["码农","技术宅","直男"]},"highlight" : {"name" : ["<p class='key' style='color:red'>阿</p><p class='key' style='color:red'>昌</p>" #需要高亮的内容被加上了em标签，这里加上了我们上面自定义的内容标签]}}]}
}

9、集成SpringBoot

官网
添加依赖

<dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

springboot2.5.4版本默认引入的es版本是7.12.1

我们需要自定义es的版本，让他跟我们使用的版本是一致的

自定义es版本

    <properties><java.version>1.8</java.version><repackage.classifier/><spring-native.version>0.10.3</spring-native.version><elasticsearch.version>7.14.1</elasticsearch.version> #自定义版本依赖</properties>

ElasticsearchRestClientAutoConfiguration自动配置类中导入了三个类

RestClientBuilderConfiguration、RestHighLevelClientConfiguration、RestClientSnifferConfiguration，

他们都是ElasticsearchRestClientConfigurations中的内部类

自定义配置

//es配置类
@Configuration
public class ElasticSearchClientConfig {@Beanpublic RestHighLevelClient restHighLevelClient() {RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(//如果是集群就构建多个new HttpHost("localhost", 9200, "http")//ip，端口，协议));return client;}}

编写测试类

创建索引

//创建索引
@Test
void contextLoads() throws IOException {//1、创建索引请求CreateIndexRequest request = new CreateIndexRequest("achang_index");//2、通过客户端执行请求,获得响应CreateIndexResponse response = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);System.out.println(response);//org.elasticsearch.client.indices.CreateIndexResponse@7e130706
}

索引存在

//索引是否存在
@Test
void test() throws IOException {//获取索引请求GetIndexRequest request = new GetIndexRequest("achang_index");boolean isExist = restHighLevelClient.indices().exists(request, RequestOptions.DEFAULT);System.out.println(isExist);//true|false
}

删除索引

//删除索引
@Test
void test1() throws IOException {//删除索引请求DeleteIndexRequest request = new DeleteIndexRequest("achang_index");AcknowledgedResponse response = restHighLevelClient.indices().delete(request, RequestOptions.DEFAULT);System.out.println(response);
}

添加文档

*//添加文档
@Test
void test2() throws IOException {User user = new User("阿昌",18);IndexRequest request = new IndexRequest("achang_index");// 规则 PUT /index/_doc/1request.id("1");request.timeout(TimeValue.timeValueSeconds(1));//超时时间1秒//将数据序列化request.source(JSON.toJSON(user), XContentType.JSON);//放入资源，设置请求格式为JSONIndexResponse response = restHighLevelClient.index(request, RequestOptions.DEFAULT);System.out.println(response.status());System.out.println(response.toString());
}

判断文档是否存在

//判断文档是否存在
@Test
void test3() throws IOException {GetRequest request = new GetRequest("achang_index", "1");request.fetchSourceContext(new FetchSourceContext(false));//不获取返回的 _source的上下文request.storedFields("_none_");boolean isExist = restHighLevelClient.exists(request, RequestOptions.DEFAULT);System.out.println(isExist);
}

获取文档

//获取文档
@Test
void test4() throws IOException {GetRequest request = new GetRequest("achang_index", "1");GetResponse response = restHighLevelClient.get(request, RequestOptions.DEFAULT);System.out.println(response);System.out.println(response.getSourceAsString());//获取得到的json对象s
}

更新文档

//更新文档
@Test
void test5() throws IOException {User user = new User("昌昌",18);UpdateRequest request = new UpdateRequest("achang_index", "1");request.doc(JSON.toJSON(user),XContentType.JSON);UpdateResponse response = restHighLevelClient.update(request, RequestOptions.DEFAULT);System.out.println(response);
}

删除文档

//删除文档
@Test
void test6() throws IOException {DeleteRequest request = new DeleteRequest("achang_index", "1");DeleteResponse response = restHighLevelClient.delete(request, RequestOptions.DEFAULT);System.out.println(response);
}

批量插入数据（修改，删除类似操作）

@Test
void testBulkRequest() throws IOException {BulkRequest request = new BulkRequest();//批量请求request.timeout("10s");//封装对象集合ArrayList<User> users = new ArrayList<>();users.add(new User("achang1", 21));users.add(new User("achang2", 22));users.add(new User("achang3", 23));users.add(new User("achang4", 18));users.add(new User("achang5", 19));// 批处理请求， 修改，删除，只要在这里修改相应的请求就可以for (int i = 0; i < users.size(); i++) {//根据请求这里设置crud的请求request.add(new IndexRequest("achang_index")//指定索引.id(String.valueOf(i + 1))//设置id//设置请求内容和请求格式.source(JSON.toJSONString(users.get(i)), XContentType.JSON));}//客户端批量发送请求BulkResponse bulkResponse = client.bulk(request, RequestOptions.DEFAULT);//是否失败，返回false表示成功System.out.println(bulkResponse.hasFailures());
}

查询文档

@Test
void testSearch() throws IOException {//查询请求SearchRequest searchRequest = new SearchRequest("achang_index");// 构建搜索条件SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();// 查询条件， 可以使用QueryBuilders工具类实现// QueryBuilders.termQuery 精确// QueryBuilders.matchLLQuery() 匹配所有TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("name", "achang1");// MatchAllQueryBuilder matchAllQueryBuilder = QueryBuilders.matchAllQuery();sourceBuilder.query(termQueryBuilder);sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));searchRequest.source(sourceBuilder);SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);System.out.println(JSON.toJSON(searchResponse.getHits()));System.out.println("======================================");//获取响应命中的数据，并遍历for (SearchHit documentFields : searchResponse.getHits().getHits()) {//获取到资源，并转换为map集合System.out.println(documentFields.getSourceAsMap());}
}

10、实战：模拟全文搜索-京东搜索

xml依赖

<dependencies><dependency><!--序列化反序列化工具包--><groupId>com.alibaba</groupId><artifactId>fastjson</artifactId><version>1.2.73</version></dependency><dependency><!--网页解析工具，爬虫--><groupId>org.jsoup</groupId><artifactId>jsoup</artifactId><version>1.13.1</version></dependency></dependencie>

ES配置类

@Configuration
public class ElasticSearchClientConfig {@Beanpublic RestHighLevelClient restHighLevelClient() {RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost("localhost", 9200, "http")));return client;}
}

网页解析工具类，爬虫

public class HtmlParseUtl {public static List<Content> parseJD(String keywords) throws IOException {String url = "https://search.jd.com/Search?keyword=" + keywords+"&enc=utf-8";Document document = Jsoup.parse(new URL(url), 30000);Element divElement = document.getElementById("J_goodsList");Elements lis = divElement.getElementsByTag("li");List<Content> contents = new ArrayList<>();for (Element li : lis) {String title = li.getElementsByClass("p-name").eq(0).text();// 注意：懒加载的图片String imgUrl = li.getElementsByTag("img").eq(0).attr("data-lazy-img");String price = li.getElementsByClass("p-price").eq(0).text();contents.add(new Content(title, imgUrl, price));}return contents;}
}

网页解析后的bean对象

public class Content implements Serializable {private String title;private String imgUrl;private String price;//省略get/set/有参/无参
}

controller层

@RestController
public class ContentController {@Autowiredprivate ContentService contentService;//解析关键字@GetMapping("/parse/{keyword}")public Boolean parse(@PathVariable("keyword") String keyword) throws IOException {return contentService.parseContent(keyword);}//搜索关键字分页@GetMapping("/search/{keywords}/{pageNo}/{pageSize}")public List<Map<String, Object>> search(@PathVariable("keywords") String keywords, @PathVariable("pageNo") int pageNo, @PathVariable("pageSize") int pageSize) throws IOException {return contentService.searchPageForHighlight(keywords, pageNo, pageSize);}
}

常量类

public class CommonConstant {public static final String INDEX = "test_jd_goods";
}

service层（省略service接口）

@Service
public class ContentService {//注入es客户端@Autowiredprivate RestHighLevelClient restHighLevelClient;//根据关键词解析，并放入es中public Boolean parseContent(String keywords) throws IOException {//解析关键字，并解析对象List<Content> contents = HtmlParseUtl().parseJD(keywords);//设置批量操作请求BulkRequest bulkRequest = new BulkRequest();bulkRequest.timeout("2m");//设置超时时间//创建批量插入操作for (int i = 0;i < contents.size(); i ++) {//变量每个解析出来的对象，都放入es中bulkRequest.add(new IndexRequest(CommonConstant.INDEX)//设置要放入的index索引.source(JSON.toJSONString(contents.get(i)), XContentType.JSON));}BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);return !bulk.hasFailures();}//分页查询关键词public List<Map<String, Object>> searchPage(String keywords, int pageNo, int pageSize) throws IOException {if (pageNo <= 1) {pageNo = 1;}// 条件搜索SearchRequest searchRequest = new SearchRequest(CommonConstant.INDEX);//查询请求SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();//查询条件构造器// 分页sourceBuilder.from(pageNo);sourceBuilder.size(pageSize);// 精准匹配TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("title", keywords);sourceBuilder.query(termQueryBuilder);sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));// 执行搜索searchRequest.source(sourceBuilder);//放入查询请求中SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);// 解析结果List<Map<String, Object>> list = new ArrayList<>();for (SearchHit documentField: searchResponse.getHits().getHits()) {list.add(documentField.getSourceAsMap());}return list;}// 根据关键词分页查询public List<Map<String, Object>> searchPageForHighlight(String keywords, int pageNo, int pageSize) throws IOException {if (pageNo <= 1) {pageNo = 1;}// 条件搜索SearchRequest searchRequest = new SearchRequest(CommonConstant.INDEX);SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();// 分页sourceBuilder.from(pageNo);sourceBuilder.size(pageSize);// 精准匹配TermQueryBuilder termQueryBuilder = QueryBuilders.termQuery("title", keywords);sourceBuilder.query(termQueryBuilder);sourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));// 高亮HighlightBuilder highlightBuilder = new HighlightBuilder();highlightBuilder.field("title");//设置高亮的字段highlightBuilder.requireFieldMatch(false);   // 多个高亮显示！highlightBuilder.preTags("<span style='color:red'>");//自定义高亮css前缀highlightBuilder.postTags("</span>");//自定义高亮css后缀sourceBuilder.highlighter(highlightBuilder);// 执行搜索searchRequest.source(sourceBuilder);SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);// 解析结果List<Map<String, Object>> list = new ArrayList<>();for (SearchHit documentField: searchResponse.getHits().getHits()) {//获取高亮的mapMap<String, HighlightField> highlightFields = documentField.getHighlightFields();HighlightField title = highlightFields.get("title");//获取原来的结果Map<String, Object> sourceAsMap = documentField.getSourceAsMap();// 解析高亮的字段， 将原来的字段换为我们高亮的字段即可！if (title != null) {Text[] fragments = title.fragments();//取出高亮的字段String n_title = "";for (Text text: fragments) {//封装替换成完美分装好的标签titlen_title += text;}//替换之前的title为我们高亮的titlesourceAsMap.put("title",n_title);//前端直接解析标签即可}list.add(sourceAsMap);}return list;}
}

整体效果

资料学习来源参考视频：B站狂神ES视频

Day390391392.-ElasticSearch入门 -ElasticSearch相关推荐

python elasticsearch 入门教程(二) ---全文搜索
python elasticsearch 入门教程(二) ---全文搜索截止目前的搜索相对都很简单:单个姓名,通过年龄过滤.现在尝试下稍微高级点儿的全文搜索--一项传统数据库确实很难搞定的任务. ...
Spring Data ElasticSearch入门案例
Spring Data ElasticSearch入门案例创建maven工程elasticsearch_springdata 基于maven导入坐标导入spring data elasticsea ...
ElasticSearch入门第一篇：Windows下安装ElasticSearch
这是ElasticSearch 2.4 版本系列的第一篇: ElasticSearch入门第一篇:Windows下安装ElasticSearch ElasticSearch入门第二篇:集群配置 E ...
ElasticSearch入门：Windows下安装ElasticSearch
这是ElasticSearch 2.4 版本系列的第一篇: ElasticSearch入门第一篇:Windows下安装ElasticSearch ElasticSearch入门第二篇:集群配置 E ...
ElasticSearch入门附.Net Core例子
1.什么是ElasticSearch? Elasticsearch是基于Lucene的搜索引擎.它提供了一个分布式,支持多租户的全文搜索引擎,它具有HTTP Web界面和无模式JSON文档. Elas ...
docker删除es数据_木杉入门Elasticsearch(4)：安装ES
木杉入门本系列的目标是在本地搭建一个Elasticsearch的服务集群,通过在阿里云服务器搭建FRP通道对外提供服务,为小规模应用提供一种高性价比的解决思路. 系列内容: 木杉入门Elastics ...
ElasticSearch入门教程-索引
ElasticSearch入门教程-索引在本节中,我们将向Elasticsearch添加一些索引,映射和数据.此数据将用于本教程中说明的示例中. 创建索引 PUT http://localhost: ...
虚拟机如何配置网络ip地址_木杉入门Elasticsearch(2)：虚拟机IP地址配置
木杉入门本系列的目标是在本地搭建一个Elasticsearch的服务集群,通过在阿里云服务器搭建FRP通道对外提供服务,为小规模应用提供一种高性价比的解决思路. 系列内容: 木杉入门Elastics ...
ElasticSearch入门第五篇：使用C#查询文档
网址:http://www.cnblogs.com/ljhdo/p/4550135.html 这是ElasticSearch 2.4 版本系列的第五篇: ElasticSearch入门第一篇:Win ...

Day390391392.-ElasticSearch入门 -ElasticSearch

ES