Elasticsearch 特定场景下使用
2024-05-12 03:28:14
假设一： 现在有一批互联网数据
| 信息 | 字段 |
|---|---|
| 名字 | name |
| 联系人 | listeners |
| 开始时间 | starttime |
| 结束时间 | endtime |
| 位置 | position |
| 联系内容 | content |
需求一：实时搜索到在目标时间内和目标地(方圆3千米)内所有联系过的人员
需求二：实时统计在目标时间内和目标人所有联系过的人员及次数
需求三：实时统计在目标时间内和目标人所有联系过人员的内容按时间排序
创建索引
PUT phonebills
{"mappings": {"properties": {"name": {"type": "keyword"},"listeners": {"type": "keyword"},"starttime": {"type": "date"},"endtime": {"type": "date"},"position": {"type": "geo_point"},"content": {"type": "text"}}}
}
将数据存入索引
需求一：实时搜索到在目标时间内和目标地(半径 x 千米)内所有联系过的人员
eg:搜索2019-04-20 ~ 2019-04-25时间段内北纬: 31.4775511291,东经: 118.4775511半径为3km内联系过的人
GET phonebills/_search
{"query": {"bool": {"must": [{"range": {"endtime": {"gte": "2019-04-20","lt": "2019-04-25"}}},{"range": {"starttime": {"gte": "2019-04-20","lt": "2019-04-25"}}},{"geo_distance": {"distance": "3km","position": {"lat": 31.4775511291,"lon": 118.4775511}}}]}}
}
结果
{"took" : 105,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 3,"relation" : "eq"},"max_score" : 2.0,"hits" : [{"_index" : "phonebills","_type" : "_doc","_id" : "UulM_2oB9G4CD_ESNTBF","_score" : 2.0,"_source" : {"listeners" : "颜嫣泺","name" : "柳阳依","endtime" : "2019-04-20T18:36:51.000Z","starttime" : "2019-04-20T17:41:56.000Z","position" : {"lon" : 118.48527643434377,"lat" : 31.48527643434377},"content" : "知道了我一个人好多年了"}},{"_index" : "phonebills","_type" : "_doc","_id" : "G-lN_2oB9G4CD_ESFjN5","_score" : 2.0,"_source" : {"listeners" : "姚洛恹","name" : "庄秋语","endtime" : "2019-04-24T15:41:30.000Z","starttime" : "2019-04-24T14:36:27.000Z","position" : {"lon" : 118.46458058351088,"lat" : 31.464580583510873},"content" : "你喜欢怎么？？啊"}},{"_index" : "phonebills","_type" : "_doc","_id" : "4OlN_2oB9G4CD_ESVDNc","_score" : 2.0,"_source" : {"listeners" : "尤沙秀","name" : "凌杉","endtime" : "2019-04-22T13:51:20.000Z","starttime" : "2019-04-22T13:25:48.000Z","position" : {"lon" : 118.48611739280219,"lat" : 31.486117392802186},"content" : "我朋友告诉我郭广昌被抓了"}}]}
}
java 实现
QueryBuilder startTime = QueryBuilders.rangeQuery("starttime").from("2019-04-20").to("2019-04-25").includeLower(true).includeUpper(false);QueryBuilder endTime = QueryBuilders.rangeQuery("endtime").from("2019-04-20").to("2019-04-25").includeLower(true).includeUpper(false);QueryBuilder position = QueryBuilders.geoDistanceQuery("position").point(31.4775511291, 118.4775511).distance(3, DistanceUnit.KILOMETERS);SearchRequestBuilder srb = client.prepareSearch("phonebills").setTypes("_doc");SearchResponse search = srb.setQuery(QueryBuilders.boolQuery().must(startTime).must(endTime).must(position)).setSize(1000).get();for (SearchHit item : search.getHits().getHits()) {System.out.println(item);}
需求二：实时统计在目标时间内和目标人所有联系过的人员及次数
eg:搜索2019-01-01 ~ 2019-05-25时间段内联系过的人及其次数
GET phonebills/_search
{"size": 0, "aggs": {"genres": {"terms": {"field": "listeners"}}},"query": {"bool": {"must": [{"range": {"endtime": {"gte": "2019-01-01","lt": "2019-05-25"}}},{"range": {"starttime": {"gte": "2019-01-01","lt": "2019-05-25"}}},{"term": {"name": {"value": "欧月"}}}]}}
}
结果
{"took" : 5,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 31,"relation" : "eq"},"max_score" : null,"hits" : [ ]},"aggregations" : {"genres" : {"doc_count_error_upper_bound" : 0,"sum_other_doc_count" : 16,"buckets" : [{"key" : "汪诗敏","doc_count" : 2},{"key" : "沐厝","doc_count" : 2},{"key" : "沐连鱼","doc_count" : 2},{"key" : "语川雅","doc_count" : 2},{"key" : "韩紫语","doc_count" : 2},{"key" : "乔昭宁","doc_count" : 1},{"key" : "凌杉","doc_count" : 1},{"key" : "夏洲","doc_count" : 1},{"key" : "姚海约","doc_count" : 1},{"key" : "安佑吟","doc_count" : 1}]}}
}
java
QueryBuilder startTime = QueryBuilders.rangeQuery("starttime").from("2019-01-01").to("2019-05-25").includeLower(true).includeUpper(false);QueryBuilder endTime = QueryBuilders.rangeQuery("endtime").from("2019-01-01").to("2019-05-25").includeLower(true).includeUpper(false);QueryBuilder name = QueryBuilders.termQuery("name", "欧月");SearchRequestBuilder srb = client.prepareSearch("phonebills").setTypes("_doc");SearchResponse search = srb.setQuery(QueryBuilders.boolQuery().must(startTime).must(endTime).must(name)).addAggregation(AggregationBuilders.terms("agg").field("listeners")).setSize(0).get();Terms agg = search.getAggregations().get("agg");List<? extends Terms.Bucket> buckets = agg.getBuckets();for (Terms.Bucket bucket : buckets) {System.out.println(bucket.getKey() + " " + bucket.getDocCount());}
需求三：实时统计在目标时间内和目标人所有联系过人员的内容按时间排序
eg:搜索2019-04-20 ~ 2019-04-25时间段内联系过人的内容排序
GET phonebills/_search
{"_source": ["content","starttime","listeners"],"size": 10000,"sort": [{"starttime": {"order": "desc"}}],"query": {"bool": {"must": [{"range": {"endtime": {"gte": "2019-01-01","lt": "2019-05-25"}}},{"range": {"starttime": {"gte": "2019-01-01","lt": "2019-05-25"}}},{"term": {"name": {"value": "欧月"}}}]}}
}
结果
{"took" : 5,"timed_out" : false,"_shards" : {"total" : 1,"successful" : 1,"skipped" : 0,"failed" : 0},"hits" : {"total" : {"value" : 31,"relation" : "eq"},"max_score" : null,"hits" : [{"_index" : "phonebills","_type" : "_doc","_id" : "n-lL_2oB9G4CD_ESrC7h","_score" : null,"_source" : {"listeners" : "殷小柠","starttime" : "2019-05-23T10:34:35.000Z","content" : "还要啥"},"sort" : [1558607675000]},{"_index" : "phonebills","_type" : "_doc","_id" : "4-lM_2oB9G4CD_ESEi98","_score" : null,"_source" : {"listeners" : "汪诗敏","starttime" : "2019-05-20T06:23:22.000Z","content" : "那我怎么办，孩子怎么办，再过一个月就能看出来了"},"sort" : [1558333402000]},{"_index" : "phonebills","_type" : "_doc","_id" : "DulN_2oB9G4CD_ESEjNY","_score" : null,"_source" : {"listeners" : "语川雅","starttime" : "2019-05-16T13:07:36.000Z","content" : "你不要太敏感"},"sort" : [1558012056000]},{"_index" : "phonebills","_type" : "_doc","_id" : "8OlN_2oB9G4CD_ESWTPr","_score" : null,"_source" : {"listeners" : "菊乱晓","starttime" : "2019-05-11T05:53:14.000Z","content" : "陶老爷"},"sort" : [1557553994000]},{"_index" : "phonebills","_type" : "_doc","_id" : "SOlM_2oB9G4CD_ESgzF9","_score" : null,"_source" : {"listeners" : "欧月","starttime" : "2019-05-10T20:24:55.000Z","content" : "老公"},"sort" : [1557519895000]},{"_index" : "phonebills","_type" : "_doc","_id" : "5elM_2oB9G4CD_EStDGF","_score" : null,"_source" : {"listeners" : "韩紫语","starttime" : "2019-05-07T19:37:13.000Z","content" : "如果你下面长的像茄子吗"},"sort" : [1557257833000]},{"_index" : "phonebills","_type" : "_doc","_id" : "ROlK_2oB9G4CD_ES7ywk","_score" : null,"_source" : {"listeners" : "乔昭宁","starttime" : "2019-05-04T21:14:06.000Z","content" : "来吧"},"sort" : [1557004446000]},{"_index" : "phonebills","_type" : "_doc","_id" : "pulM_2oB9G4CD_ESnzH7","_score" : null,"_source" : {"listeners" : "裴明秋","starttime" : "2019-05-04T17:02:03.000Z","content" : "带手铐的"},"sort" : [1556989323000]},{"_index" : "phonebills","_type" : "_doc","_id" : "uOlL_2oB9G4CD_EStC4Y","_score" : null,"_source" : {"listeners" : "庄秋言","starttime" : "2019-04-27T06:14:28.000Z","content" : "怎么就不合适了"},"sort" : [1556345668000]},{"_index" : "phonebills","_type" : "_doc","_id" : "welL_2oB9G4CD_ESZi3C","_score" : null,"_source" : {"listeners" : "语川雅","starttime" : "2019-04-25T02:23:02.000Z","content" : "我在想你要老婆"},"sort" : [1556158982000]},{"_index" : "phonebills","_type" : "_doc","_id" : "ZelK_2oB9G4CD_ES-ixr","_score" : null,"_source" : {"listeners" : "殷小室","starttime" : "2019-04-20T15:36:46.000Z","content" : "从话费，发照片，从来都是裸照"},"sort" : [1555774606000]},{"_index" : "phonebills","_type" : "_doc","_id" : "s-lL_2oB9G4CD_ESFCwf","_score" : null,"_source" : {"listeners" : "韩紫语","starttime" : "2019-04-16T15:10:47.000Z","content" : "你喜欢怎么做爱啊"},"sort" : [1555427447000]},{"_index" : "phonebills","_type" : "_doc","_id" : "kelM_2oB9G4CD_ESmTGc","_score" : null,"_source" : {"listeners" : "宋晓茶","starttime" : "2019-03-29T12:18:21.000Z","content" : "啊，那不用了"},"sort" : [1553861901000]},{"_index" : "phonebills","_type" : "_doc","_id" : "YulN_2oB9G4CD_ESKzOh","_score" : null,"_source" : {"listeners" : "秦不语","starttime" : "2019-03-16T01:45:50.000Z","content" : "不好意思，打扰你睡觉了"},"sort" : [1552700750000]},{"_index" : "phonebills","_type" : "_doc","_id" : "felL_2oB9G4CD_ESUi1J","_score" : null,"_source" : {"listeners" : "姚海约","starttime" : "2019-03-14T04:18:10.000Z","content" : "是啊，人生苦短 要好好享受快乐"},"sort" : [1552537090000]},{"_index" : "phonebills","_type" : "_doc","_id" : "oelL_2oB9G4CD_ESDiyI","_score" : null,"_source" : {"listeners" : "沐厝","starttime" : "2019-03-05T07:15:32.000Z","content" : "妹妹喜欢茄子还是黄瓜"},"sort" : [1551770132000]},{"_index" : "phonebills","_type" : "_doc","_id" : "fOlM_2oB9G4CD_ES5DLA","_score" : null,"_source" : {"listeners" : "安佑吟","starttime" : "2019-03-02T05:33:34.000Z","content" : "x6的"},"sort" : [1551504814000]},{"_index" : "phonebills","_type" : "_doc","_id" : "6elN_2oB9G4CD_ESBzKr","_score" : null,"_source" : {"listeners" : "沐厝","starttime" : "2019-03-01T23:04:51.000Z","content" : "你是不是第一次让人家给你充话费"},"sort" : [1551481491000]},{"_index" : "phonebills","_type" : "_doc","_id" : "s-lM_2oB9G4CD_ES9jKk","_score" : null,"_source" : {"listeners" : "凌杉","starttime" : "2019-02-25T15:54:36.000Z","content" : "过几天，我叫我的同事也来，你也来"},"sort" : [1551110076000]},{"_index" : "phonebills","_type" : "_doc","_id" : "4-lL_2oB9G4CD_EScS2v","_score" : null,"_source" : {"listeners" : "汪诗敏","starttime" : "2019-02-11T23:59:34.000Z","content" : "在干嘛呢"},"sort" : [1549929574000]},{"_index" : "phonebills","_type" : "_doc","_id" : "7OlL_2oB9G4CD_ESdC2q","_score" : null,"_source" : {"listeners" : "楚羡冰","starttime" : "2019-02-03T12:06:03.000Z","content" : "非常想"},"sort" : [1549195563000]},{"_index" : "phonebills","_type" : "_doc","_id" : "fOlL_2oB9G4CD_ESUS3_","_score" : null,"_source" : {"listeners" : "左让羽","starttime" : "2019-02-03T05:47:40.000Z","content" : "知道了我一个人好多年了"},"sort" : [1549172860000]},{"_index" : "phonebills","_type" : "_doc","_id" : "hulL_2oB9G4CD_ESpS5t","_score" : null,"_source" : {"listeners" : "闵喑彤","starttime" : "2019-01-30T04:49:59.000Z","content" : "我也不想出名啊"},"sort" : [1548823799000]},{"_index" : "phonebills","_type" : "_doc","_id" : "2elL_2oB9G4CD_ESbi3M","_score" : null,"_source" : {"listeners" : "沐连鱼","starttime" : "2019-01-26T01:09:10.000Z","content" : "视频就传出来"},"sort" : [1548464950000]},{"_index" : "phonebills","_type" : "_doc","_id" : "BulM_2oB9G4CD_ESHDDZ","_score" : null,"_source" : {"listeners" : "秦采桑","starttime" : "2019-01-24T19:46:05.000Z","content" : "发誓，不信你跟我叫他过来解释"},"sort" : [1548359165000]},{"_index" : "phonebills","_type" : "_doc","_id" : "HOlM_2oB9G4CD_ESIzA8","_score" : null,"_source" : {"listeners" : "言飘舞","starttime" : "2019-01-23T04:19:33.000Z","content" : "嗯亲爱的我和宝贝会乖乖等你的"},"sort" : [1548217173000]},{"_index" : "phonebills","_type" : "_doc","_id" : "hOlM_2oB9G4CD_ES5zIP","_score" : null,"_source" : {"listeners" : "梦临韵","starttime" : "2019-01-21T20:52:31.000Z","content" : "这么便宜"},"sort" : [1548103951000]},{"_index" : "phonebills","_type" : "_doc","_id" : "IOlL_2oB9G4CD_ES1C8J","_score" : null,"_source" : {"listeners" : "蓝晴嫣","starttime" : "2019-01-04T08:31:41.000Z","content" : "喜欢从背后"},"sort" : [1546590701000]},{"_index" : "phonebills","_type" : "_doc","_id" : "uulM_2oB9G4CD_ESVjAl","_score" : null,"_source" : {"listeners" : "沐连鱼","starttime" : "2019-01-03T11:45:19.000Z","content" : "哈哈哈哈哈哈多么痛的领悟"},"sort" : [1546515919000]},{"_index" : "phonebills","_type" : "_doc","_id" : "relM_2oB9G4CD_ESojFy","_score" : null,"_source" : {"listeners" : "夏洲","starttime" : "2019-01-03T07:30:53.000Z","content" : "我们一家在老妈处吃完饭"},"sort" : [1546500653000]},{"_index" : "phonebills","_type" : "_doc","_id" : "culN_2oB9G4CD_ESMTNN","_score" : null,"_source" : {"listeners" : "施敏依","starttime" : "2019-01-01T18:44:27.000Z","content" : "过几天，我叫我的同事也来，你也来"},"sort" : [1546368267000]}]}
}
java
QueryBuilder startTime = QueryBuilders.rangeQuery("starttime").from("2019-01-01").to("2019-05-25").includeLower(true).includeUpper(false);QueryBuilder endTime = QueryBuilders.rangeQuery("endtime").from("2019-01-01").to("2019-05-25").includeLower(true).includeUpper(false);QueryBuilder name = QueryBuilders.termQuery("name", "欧月");SearchRequestBuilder srb = client.prepareSearch("phonebills").setTypes("_doc");String[] includes = new String[]{"content", "starttime", "listeners"};String[] excludes = new String[]{};SearchResponse search = srb.setQuery(QueryBuilders.boolQuery().must(startTime).must(endTime).must(name)).addSort("starttime", SortOrder.DESC).setFetchSource(includes, excludes).setSize(1000).get();for (SearchHit item : search.getHits().getHits()) {System.out.println(item);}
Elasticsearch 特定场景下使用相关推荐
- 特定场景下Yolo改进算法：Poly-Yolo
论文名称:Poly-YOLO: higher speed, more precise detection and instance segmentation for YOLOv3 论文地址:https ...
- [Camera Drv]开video dynamic framerate，特定场景下video encode时会闪屏 - MTK物联网在线解答 - 技术论坛
[Camera Drv]开video dynamic frame rate,特定场景下video encode时会闪屏 1. 开 video dynamic frame rate ,环境 BV 在 d ...
- 特定场景下取代if-else和switch的方式
look-up表代替if-else 比如某平台的信用分数评级: 超过700-950,信用极好, 650-700信用优秀, 600-650信用良好, 550-600信用中等, 350-550信用较差. ...
- 关于如何解决特定场景下WPF4.0中“XamlWriter.Save序列化限制”问题的一种思路
问题背景 笔者最近使用ArcGIS Server WPF API 开发一个能够实现所见即所得的基于Xaml的图层符号编辑功能.这个功能主要目的能够让用户以Xaml的形式来定义图层符号,并能够 ...
- 特定场景下SQL的优化
1.大表的数据修改最好分批处理. 1000万行的记录表中删除更新100万行记录,一次只删除或更新5000行数据.每批处理完成后,暂停几秒中,进行同步处理. 2.如何修改大表的表结构. 对表的列的字段类 ...
- 神策数据成林松：数据智能在业务场景下的应用（附 PPT 下载）
在神策 2020 数据驱动用户大会「上海站」现场,神策数据业务咨询师成林松分享了<数据智能在业务场景下的应用>的演讲.(文末附 PPT 下载地址) 本文根据其演讲内容整理,数据均为虚拟. ...
- swoole mysql 并发_如何用Swoole测试MySQL在特定SQL下的并发性能
场景描述 从全文检索或者缓存中获取ID,根据ID查询数据库获取基础信息,进行页面展示 SQL:select * from table where id in(id1,id2,id3...id40) 此 ...
- [Android][sensor]确认sensor唤醒源：查看某一时刻\某个场景下，某个sensor被哪个上层apk调用
某一时刻\某个场景下 我们有时候需要在某一特定场景下查看某个sensor被什么apk调用. 比如在sim卡热插拔时,发现acc被唤醒(软件排查发现:依次去掉每个sensor器件的编译),这会导致在si ...
- 一文全览，深度学习时代下，复杂场景下的 OCR 如何实现？
2020-02-18 05:35:34 Source: kurzweilai.net 作者 | 北京矩视智能科技有限公司 责编 | 贾伟 文本是人类最重要的信息来源之一,自然场景中充满了形形色色的文字 ...
- 火眼金睛算法，教你海量短文本场景下去重
2019独角兽企业重金招聘Python工程师标准>>> 本文由QQ大数据发表 最朴素的做法 在大多数情况下,大量的重复文本一般不会是什么好事情,比如互相抄袭的新闻,群发的垃圾短信,铺 ...
最新文章
- linux 无线网络配置工具wpa_supplicant与wireless-tools
- html figure标签并排显示,html figure标签怎么用
- Build OpenVSwitch and OVN
- PyTorch基础（12）-- torch.nn.BatchNorm2d()方法
- Angular应用里input字段后面的_ngcontent-hqi是什么含义
- 【软件工程】IS的三级管理
- 熟悉网络层IP协议和数据链路层
- 麦马计算机专业对化学的要求,2018麦克马斯特大学最新入学要求＋热门专业全解析...
- 三种Windows版本下教你如何卸载Oracle
- 优秀程序设计的原则（可以多读读）
- 有道翻译与VS2010滚动栏自动反弹冲突问题
- linux中posix共享内存,Linux 共享内存（POSIX）
- 问题2：无法打开包括文件：“windows.h”:No such file or directory
- Hive调优全方位指南（推荐收藏）
- 阅读《原则》有感之工作原则
- 顺丰该不该开除删库的运维工程师？
- chatGPT发送图片的方法
- 每日一诗词 —— 越人歌
- [SilkyBible] XviD系列-9
- wx.showModal(模态框)的相关设置