kibana

是一一个针对Elasticsearch的开源分析及可视化平台,用来搜索、查看交互存储在Elasticsearch索引中的数据。使用Kibana ,可以通过各种图表进行高级数据分析及展示。Kibana让海量数据更容易理解。它操作简单,基于浏览器的用户界面可以快速创建仪表板( dashboard )实时显示Elasticsearch查询动态。设置Kibana非常简单。无需编码或者额外的基础架构,几分钟内就可以完成Kibana安装并启动Elasticsearch索引监测。

kibana的安装和使用：

https://blog.csdn.net/qq_18769269/article/details/80843810

ES查询的两种方式

基础知识

命令

命令	url	解释
put	/索引名称/类型名称/文档ID	创建文档（指定文档ID）
POST	/索引名称/索引类型	创建文档（随机文档ID）
POST	/索引名称/类型名称/文档id/_update	修改文档
POST	/索引名称/类型名称/_search	查询数据
DELETE	/索引名称/类型名称/文档id	删除文档/或者索引
GET	/索引名称/类型名称/文档id	查询文档通过文档ID

字段类型

类型	对应类型	说明
字符串	text keyword	text自动分词，keyword全文匹配
整型	byte short integer long
浮点型	float double half_float scaled_float
日期	date
布尔	boolean
二进制	binary
范围	range
数组	array
对象	object
嵌套	nested
ip	ip (IPv4 和 IPv6 地址)

1、查询字符串搜索

GET alias_productbatches/_search?q=id:1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3{"took" : 1,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 1,"max_score" : 10.914281,"hits" : [{"_index" : "productbatches_v2","_type" : "_doc","_id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3","_score" : 10.914281,"_source" : {"id" : "1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3","num" : "202000004107210719150940932","product_id" : "56e9dab5-8ff8-410a-8e2d-995a98eb54b1","store_id" : "1471d41e-c30c-4ffb-90ba-98febbfae394","store_product_id" : "5eaf6b02-c727-4bf7-b9af-15f11421f900",}}]}
}

2、结构化查询(单字段查询,不能多字段组合查询)

GET alias_productbatches/_search
{"query":{"match":{"id":"1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3"}}
}

3、match系列之操作

造数：PUT test1/doc/1
{"title": "中国是世界上人口最多的国家","desc": "china is the most people in the world"
}
PUT test1/doc/2
{"title": "美国是世界上军事实力最强大的国家","desc": "The United States is the most powerful military country in the world"
}
PUT test1/doc/3
{"title": "北京是中国的首都","desc": "Beijing is the capital of China"
}

match: 查询匹配key的values值GET test1/doc/_search
{"query":{"match":{"title":"中国"}}
}结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 0.68324494,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.68324494,"_source" : {"title" : "中国是世界上人口最多的国家"}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 0.5753642,"_source" : {"title" : "北京是中国的首都"}},{"_index" : "test1","_type" : "doc","_id" : "2","_score" : 0.39556286,"_source" : {"title" : "美国是世界上军事实力最强大的国家"}}]}
}匹配id=1e8ca3d2-5515-45b9-9a7d-4d9ac3bcc0d3match查询中文时，会把中文拆分后进行匹配查询，如需要则使用短语查询match_phraseGET test1/doc/_search
{"query":{"match_phrase":{"title":"中国"}}
}结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 2,"max_score" : 0.5753642,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.5753642,"_source" : {"title" : "中国是世界上人口最多的国家"}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 0.5753642,"_source" : {"title" : "北京是中国的首都"}}]}
}slop:相当于正则中的中国.*?世界。这个间隔默认为0，2代表中国与世界中间数字在两个及以下
GET test1/doc/_search
{"query":{"match_phrase": {"title": {"query": "中国世界","slop":2}}}
}结果：
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 1,"max_score" : 0.7445889,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.7445889,"_source" : {"title" : "中国是世界上人口最多的国家"}}]}
}match_phrase_prefix（最左前缀查询）智能搜索--以什么开头（主要是英文）
GET test1/doc/_search
{"query":{"match_phrase_prefix": {"desc": "china is"}}
}结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 1,"max_score" : 0.5753642,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.5753642,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most people in the world"}}]}
}max_expansions 参数理解 前缀查询会非常的影响性能，要对结果集进行限制，就加上这个参数。GET test1/doc/_search
{"query": {"match_phrase_prefix": {"desc": {"query": "bea","max_expansions":1}}}
}

match系列之multi_match（多字段查询)

multi_match是要在多个字段中查询同一个关键字除此之外，mulit_match甚至可以当做match_phrase和match_phrase_prefix使用，只需要指定type类型即可

GET test1/doc/_search
{"query": {"multi_match": {"query": "中国","fields": ["title"]}}
}同上第一个match查询当设置属性 type:phrase 时 等同于 短语查询
GET test1/doc/_search
{"query": {"multi_match": {"query": "中国","fields": ["title"],"type": "phrase"}}
}当设置属性 type:phrase_prefix时 等同于 最左前缀查询
GET test1/doc/_search
{"query": {"multi_match": {"query": "china is","fields": ["desc"],"type": "phrase_prefix"}}
}

四、ES的排序查询

sort：对字段进行排序

GET test1/doc/_search
GET test1/doc/_search
{"query": {"match_all": {}},"sort": [{"title.keyword": {"order": "asc"}}]
}为什么不能直接title呢？
查看mapping：
{"test1" : {"mappings" : {"doc" : {"properties" : {"desc" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}},"title" : {"type" : "text","fields" : {"keyword" : {"type" : "keyword","ignore_above" : 256}}}}}}}
}对text进行排序需要对字段索引两次，一次索引分词（用于搜索）一次索引不分词（用于排序）

五、ES的分页查询

from：从哪开始查 size：返回几条结果

GET test1/doc/_search
{"query": {"match": {"title": "中国"}},"from": 0,"size": 2
}结果：{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 0.68324494,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.68324494,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most people in the world"}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 0.5753642,"_source" : {"title" : "北京是中国的首都","desc" : "Beijing is the capital of China"}}]}
}

六、ES的bool查询 (must、should)

must (must字段对应的是个列表，也就是说可以有多个并列的查询条件，一个文档满足各个子条件后才最终返回)

GET test1/doc/_search
{"query": {"bool": {"must": [{"match": {"title": "中国"}}]}}
}结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 0.68324494,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.68324494,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most people in the world"}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 0.5753642,"_source" : {"title" : "北京是中国的首都","desc" : "Beijing is the capital of China"}},{"_index" : "test1","_type" : "doc","_id" : "2","_score" : 0.39556286,"_source" : {"title" : "美国是世界上军事实力最强大的国家","desc" : "The United States is the most powerful military country in the world"}}]}
}多个条件：
GET test1/doc/_search
{"query": {"bool": {"must": [{"match": {"title": "中国"}},{"match_phrase_prefix": {"desc": "china is"}}]}}
}结果：
{"took" : 13,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 1,"max_score" : 1.258609,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 1.258609,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most people in the world"}}]}
}

should (只要符合其中一个条件就返回)

GET test1/doc/_search
{"query": {"should": {"must": [{"match": {"title": "中国"}},{"match_phrase_prefix": {"desc": "mytest"}}]}}
}结果和查询条件满足中国的结果一样

filter：满足过滤条件

filter(条件过滤查询，过滤条件的范围用range表示gt表示大于、lt表示小于、gte表示大于等于、lte表示小于等于)

GET test1/_search
{"query": {"bool": {"must": [{"range": {"time_create": {"gte": "1626667200000","lte": "1626710400000"}}}]}}}

boost：提权，控制每个查询子句的相对权重，该值默认为1。一个大于1的boost会增加该查询子句的相对权重

可以参考：https://www.jianshu.com/p/98888942e737

minimum_should_match：最小匹配度

参考：https://blog.csdn.net/xiao_jun_0820/article/details/51095521

must_not：不等于

七、ES之查询结果过滤

_source

GET test1/doc/_search
{"query": {"match": {"title": "中国"}},"_source": ["title"]
}结果：
{"took" : 1,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 0.68324494,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 0.68324494,"_source" : {"title" : "中国是世界上人口最多的国家"}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 0.5753642,"_source" : {"title" : "北京是中国的首都"}},{"_index" : "test1","_type" : "doc","_id" : "2","_score" : 0.39556286,"_source" : {"title" : "美国是世界上军事实力最强大的国家"}}]}
}=======
次数最终的结果没有展示desc字段

八、ES之精确查询与模糊查询

term:查询查找包含文档精确的倒排索引指定的词条。也就是精确查找。

term和match的区别是：match是经过analyer的，也就是说，文档首先被分析器给处理了。根据不同的分析器，分析的结果也稍显不同，然后再根据分词结果进行匹配。term则不经过分词，它是直接去倒排索引中查找了精确的值了。

GET test1/doc/_search
{"query": {"term": {"title": "中国"}},"_source": ["title"]
}
===如果这样查询，就无法查出结果GET test1/doc/_search
{"query": {"term": {"title.keyword": "中国是世界上人口最多的国家"}}
}======
这样查询title为“中国是世界上人口最多的国家”的数据,
为什么要加keyword呢，因为索引的mapping title是text:
"keyword" : {"type" : "keyword","ignore_above" : 256}

多个term查询

查询方式一：
GET test1/doc/_search
{"query": {"bool": {"must": [{"term": {"title.keyword": {"value": "中国是世界上人口最多的国家"}}},{"term": {"desc.keyword": {"value": "china is the most people in the world"}}}]}}
}方式二：
GET test1/doc/_search
{"query": {"bool": {"must": [{"terms": {"title.keyword": ["中国是世界上人口最多的国家","北京是中国的首都"]}}]}}
}

九、ES的聚合查询avg、max、min、sum

聚合函数的使用，一定是先查出结果，然后对结果使用聚合函数做处理

avg：求平均

max：最大值

min：最小值

sum：求和

GET test1/doc/_search
{"query": {"bool": {"must": []}},"aggs": {"my_aggs": {"avg": {"field": "score"}}}
}=====
查询结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 1.0,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "2","_score" : 1.0,"_source" : {"title" : "美国是世界上军事实力最强大的国家","desc" : "The United States is the most powerful military country in the world","score" : 2.5}},{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 1.0,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most populous country in the world","score" : 9.5}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 1.0,"_source" : {"title" : "北京是中国的首都","desc" : "Beijing is the capital of China","score" : 5.5}}]},"aggregations" : {"my_aggs" : {"value" : 5.833333333333333}}
}=========
query:先查询结果
aggs：对查询出来的结果进行聚合，my_aggs是聚合的别名 avg是聚合类型，field的值是聚合的字段
其他聚合方式一样
GET test1/doc/_search
{"query": {"bool": {"must": []}},"aggs": {"my_aggs": {"max": {"field": "score"}}}
}

十、ES的分组查询

在aggs的中，使用range来做分组，field是以age为分组，分组使用ranges来做，from和to是范围

GET test1/doc/_search
{"query": {"bool": {"must": []}},"aggs": {"my_aggs_group": {"range": {"field": "score","ranges": [{"from": 1.0,"to": 5.0},{"from": 5.0,"to": 10.0}]},"aggs": {"my_agg": {"avg": {"field": "score"}}}}}
}==结果：
{"took" : 0,"timed_out" : false,"_shards" : {"total" : 5,"successful" : 5,"skipped" : 0,"failed" : 0},"hits" : {"total" : 3,"max_score" : 1.0,"hits" : [{"_index" : "test1","_type" : "doc","_id" : "2","_score" : 1.0,"_source" : {"title" : "美国是世界上军事实力最强大的国家","desc" : "The United States is the most powerful military country in the world","score" : 2.5}},{"_index" : "test1","_type" : "doc","_id" : "1","_score" : 1.0,"_source" : {"title" : "中国是世界上人口最多的国家","desc" : "china is the most populous country in the world","score" : 9.5}},{"_index" : "test1","_type" : "doc","_id" : "3","_score" : 1.0,"_source" : {"title" : "北京是中国的首都","desc" : "Beijing is the capital of China","score" : 5.5}}]},"aggregations" : {"my_aggs_group" : {"buckets" : [{"key" : "1.0-5.0","from" : 1.0,"to" : 5.0,"doc_count" : 1,"my_agg" : {"value" : 2.5}},{"key" : "5.0-10.0","from" : 5.0,"to" : 10.0,"doc_count" : 2,"my_agg" : {"value" : 7.5}}]}}
}

ES数据的操作（二）相关推荐

【MySQL 数据的操作二】
目录二.更新数据记录 2.1更新特定的数据记录 2.2更新所有数据记录三.删除数据记录 3.1删除特定数据记录 3.2删除所有数据记录生词表二.更新数据记录可以通过 UPDATE 语句来实现 ...
python更新es数据_python操作es增删改查
1.查询(search) # 获取案例库信息 @app.route('/get_dcn_cases', methods=['GET', 'POST']) def get_dcn_cases(): # ...
【MySQL数据的操作一】
目录一.插入数据记录 1.1插入多条完整数据记录 1.2插入多条部分记录 1.3 插入一条记录 1.4 不指定字段插入完整记录 1.4 插入来自其他表的数据生词表一.插入数据记录在MySQL软 ...
2021年大数据HBase（二）：HBase集群安装操作
全网最详细的大数据HBase文章系列,强烈建议收藏加关注! 新文章都已经列出历史文章目录,帮助大家回顾前面的知识重点. 目录系列历史文章前言 HBase集群安装操作一.上传解压HBase安装包 ...
大数据分析必须要会的数据预处理操作（二）！！！
数学建模美赛大数据题必须要会的数据预处理操作(二)!!! 文章目录数学建模美赛大数据题必须要会的数据预处理操作(二)!!! 创建DataFrame 数据显示限制 Series操作创建Series ...
45-网上商城数据库-商品分类数据操作(二)
45-网上商城数据库-商品分类数据操作(二) 项目描述在电子商务兴起的大环境下,建立利用互联网开拓销售渠道,帮助企业及时调整商品结构,协助经销商打开货源的信息门户成为解决信息流通不畅的有效方案,电子 ...
【大数据实战项目二】Spark环境和Mongo、ES数据库安装，以及数据库与Spark，Python联动
Spark和Mongodb软件安装与python交互测试 3.1 python处理文件 3.2 搭建Spark开发环境和测试 3.3 搭建Mongodb和ES数据库及测试 3.3.1 Mongodb安 ...
python图像数据是几维数据_Python图像处理库PIL的ImagePath模块被用于存储和操作二维向量数据...
ImagePath模块被用于存储和操作二维向量数据.Path对象会被传递到ImageDraw模块中. 一.ImagePath模块的函数 1. Path 定义:ImagePath.Path(coordi ...
Arduino UNO AT24C32进行字符串数据读写操作（二）
Arduino UNO AT24C32进行字符串数据读写操作(二) 相关篇<Arduino UNO AT24C32进行字符串数据读写[优化篇]> 本篇在读写AT24C32的时候,采用移位运 ...
35-博客网站数据库-博文信息数据操作(二)
35-博客网站数据库-博文信息数据操作(二) 项目描述当今网上微博.博客发布信息已经成为主要的信息发布.传播的系统,如何对这些数据进行管理,本项目主要是对博客网站中的博文分类信息表.博文信息表进行操 ...

ES数据的操作（二）