Java调用ElasticSearch 7.2.1 保存、统计、多字段分组聚合
目录
maven引用
配置
配置类
保存数据方法
参照官方的引用方式会报错
分组统计
查询后再统计
多字段分组聚合
maven引用
注意版本与es版本一致
<dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>7.2.1</version><exclusions><exclusion><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId></exclusion></exclusions>
</dependency><dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>7.2.1</version>
</dependency>
配置
spring:data:elasticsearch:ip: 10.0.197.198port: 9200scheme: http
配置类
import lombok.Data;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;@Configuration
@ConfigurationProperties(prefix = "spring.data.elasticsearch")
@Data
public class ElasticsearchConfig {private String ip;private Integer port;private String scheme;public RestHighLevelClient getClient() {RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(new HttpHost(ip, port, scheme)));return client;}
}
保存数据方法
public Boolean setPageAccess(Map map) {try(RestHighLevelClient client= elasticsearchConfig.getClient()) {// 1、创建索引请求IndexRequest indexRequest = new IndexRequest("test"); //文档idindexRequest.id(StringUtil.getUUID());map.put("time",new Date());// 2、准备文档数据// 方式一:直接给JSON串String jsonString = JSON.toJSONString(map);indexRequest.source(jsonString, XContentType.JSON);//4、发送请求IndexResponse indexResponse = null;try {// 同步方式indexResponse = client.index(indexRequest, RequestOptions.DEFAULT);} catch (ElasticsearchException e) {// 捕获,并处理异常//判断是否版本冲突、create但文档已存在冲突if (e.status() == RestStatus.CONFLICT) {log.error("冲突了\n" + e.getDetailedMessage());}log.error("索引异常", e);return false;}}catch (Exception e){e.printStackTrace();return false;}return true;
}
参照官方的引用方式会报错
java.lang.NoClassDefFoundError: org/elasticsearch/common/xcontent/DeprecationHandler
解决办法:排除旧的es引用
<dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>7.2.1</version><exclusions><exclusion><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId></exclusion></exclusions>
</dependency><dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>7.2.1</version>
</dependency>
分组统计
注意:分组如果在字符串字段上,需要建立字段对应的.keyword字段,该字段支持聚合处理,直接用字符串字段会报错。
try(RestHighLevelClient client= elasticsearchConfig.getClient()) {// 1、创建search请求//SearchRequest searchRequest = new SearchRequest();SearchRequest searchRequest = new SearchRequest(INDEX_PAGEACCESS);// 2、用SearchSourceBuilder来构造查询请求体 ,请仔细查看它的方法,构造各种查询的方法都在这。SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();//sourceBuilder.query();sourceBuilder.size(0);//加入聚合//字段值项分组聚合TermsAggregationBuilder aggregation = AggregationBuilders.terms("by_method").field("method.keyword").order(BucketOrder.aggregation("count", true));//计算每组的平均balance指标aggregation.subAggregation(AggregationBuilders.count("count").field("_sessionid"));sourceBuilder.aggregation(aggregation);searchRequest.source(sourceBuilder);//3、发送请求SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);//4、处理响应//搜索结果状态信息if(RestStatus.OK.equals(searchResponse.status())) {// 获取聚合结果Aggregations aggregations = searchResponse.getAggregations();Terms byAgeAggregation = aggregations.get("by_method");log.info("aggregation by_age 结果");log.info("docCountError: " + byAgeAggregation.getDocCountError());log.info("sumOfOtherDocCounts: " + byAgeAggregation.getSumOfOtherDocCounts());log.info("------------------------------------");for(Terms.Bucket buck : byAgeAggregation.getBuckets()) {log.info("key: " + buck.getKeyAsString());log.info("docCount: " + buck.getDocCount());log.info("docCountError: " + buck.getDocCountError());//取子聚合ParsedValueCount averageBalance = buck.getAggregations().get("count");log.info("average_balance: " + averageBalance.getValue());log.info("------------------------------------");}//直接用key 来去分组/*Bucket elasticBucket = byCompanyAggregation.getBucketByKey("24");Avg averageAge = elasticBucket.getAggregations().get("average_age");double avg = averageAge.getValue();*/}return new ArrayList();
}catch (Exception e){e.printStackTrace();return null;
}
查询后再统计
try(RestHighLevelClient client= elasticsearchConfig.getClient()) {// 1、创建search请求//SearchRequest searchRequest = new SearchRequest();SearchRequest searchRequest = new SearchRequest(INDEX_PAGEACCESS);// 2、用SearchSourceBuilder来构造查询请求体 ,请仔细查看它的方法,构造各种查询的方法都在这。SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();//构造QueryBuilderQueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("method.keyword", "web_event_srv.upload");sourceBuilder.query(matchQueryBuilder);sourceBuilder.size(0);//加入聚合//字段值项分组聚合TermsAggregationBuilder aggregation = AggregationBuilders.terms("by_fngroup").field("fngroup.keyword").order(BucketOrder.aggregation("count", true));//计算每组的平均balance指标aggregation.subAggregation(AggregationBuilders.count("count").field("sessionid"));sourceBuilder.aggregation(aggregation);searchRequest.source(sourceBuilder);//3、发送请求SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);//4、处理响应//搜索结果状态信息List<Map> result=new ArrayList<>();if(RestStatus.OK.equals(searchResponse.status())) {// 获取聚合结果Aggregations aggregations = searchResponse.getAggregations();Terms byAgeAggregation = aggregations.get("by_fngroup");for(Terms.Bucket buck : byAgeAggregation.getBuckets()) {Map map=new HashMap();map.put("name", buck.getKeyAsString());//取子聚合ParsedValueCount averageBalance = buck.getAggregations().get("count");map.put("count",averageBalance.getValue());result.add(map);}}return result;
}catch (Exception e){e.printStackTrace();return null;
}
多字段分组聚合
按fngroup和user_id分组聚合
public List selectModuleCount(Map param) {try(RestHighLevelClient client= elasticsearchConfig.getClient()) {// 1、创建search请求//SearchRequest searchRequest = new SearchRequest();SearchRequest searchRequest = new SearchRequest(INDEX_PAGEACCESS);// 2、用SearchSourceBuilder来构造查询请求体 ,请仔细查看它的方法,构造各种查询的方法都在这。SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();sourceBuilder.size(0);//加入聚合//字段值项分组聚合TermsAggregationBuilder aggregation = AggregationBuilders.terms("by_fngroup").script(new Script("doc['fngroup.keyword'] +'#'+doc['user_id']"))//.field("fngroup.keyword").size(Integer.MAX_VALUE).order(BucketOrder.aggregation("count", true));//计算每组的平均balance指标aggregation.subAggregation(AggregationBuilders.count("count").field("sessionid"));sourceBuilder.aggregation(aggregation);searchRequest.source(sourceBuilder);//3、发送请求SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);//4、处理响应//搜索结果状态信息List<Map> result=new ArrayList<>();if(RestStatus.OK.equals(searchResponse.status())) {// 获取聚合结果Aggregations aggregations = searchResponse.getAggregations();Terms byAgeAggregation = aggregations.get("by_fngroup");for(Terms.Bucket buck : byAgeAggregation.getBuckets()) {Map map=new HashMap();String[] arr= buck.getKeyAsString().split("#");map.put("module",arr[0].replace("[","").replace("]",""));map.put("user_id",arr[1].replace("[","").replace("]",""));//取子聚合ParsedValueCount averageBalance = buck.getAggregations().get("count");map.put("count",averageBalance.getValue());result.add(map);}}return result;}catch (Exception e){e.printStackTrace();return null;}}
Java调用ElasticSearch 7.2.1 保存、统计、多字段分组聚合相关推荐
- java调用ffmpeg把rtsp视频流保存为MP4文件
前言:最近需要把rtsp的视频流保存为MP4文件(就是录制直播流).刚开始用的javacv的FFmpegFrameGrabber和FFmpegFrameRecorder,但是声音流和视频流无法调整,声 ...
- Java调用Google Analytics API实现网站统计
首先,申请Google帐号和Google Analytics服务,并将统计代码放入你想统计的网站中一段时间,确保你的Google Analytics中已有数据. 在Google Analytics中, ...
- java调用手机麦克风录音以及保存音频文件到服务器
1.应用场景:手机app内录制不超过60秒的录音,进行保存,可以实现播放 实现过程 (1).前端调取手机麦克风,录制音频,转成base64文件传给后端 (2).后端将前端传过来的base64文件进行解 ...
- java调用matlab绘图
环境: 系统:kali linux 2017 java: 1.8 matlab:R2016b idea:2016 1.编写matlab脚本: drawplot.m %画图 自定义函数drawplot ...
- java调用摄像头保存到图库_Java调用摄像头并拍摄保存
Java调用摄像头并拍摄保存. 找到一篇关于java调用硬件(摄像头),一篇挺有意思的文章,索性转一下: ------------------------------------- 转自:http:/ ...
- Java调用ffmpeg进行视频.H264抽帧,并保存为图片
Java调用ffmpeg进行视频.H264抽帧,并保存为图片 1. 需求 2. 解决 3. 源码 参考 1. 需求 对视频 D:\data\01-test.H264进行抽帧并保存为图片,图片命名为1. ...
- java调用保存好的神经网络模型
1.训练模型并保存 import pandas as pd from sklearn.preprocessing import LabelEncoder import numpy as np from ...
- java中抓拍图像_JavaCV调用摄像头并抓拍图片保存到本地
添加依赖 org.bytedeco javacv-platform 1.4.1 org.bytedeco.javacpp-presets opencv-platform 3.4.1-1.4.1 jun ...
- 初认识Java调用SPSS统计软件
最近做了一个医学院的项目,里面有一个新的技术实现我没有接触过,就是用Java调用SPSS来进行独立样本T检验,判断影响因素是否具有统计学意义.在SPSS官网找了好久,自己也在网上搜索了好久,发现关于在 ...
最新文章
- 苹果支付和ios安全 - 你需要知道的
- Keil Debug(printf) Viewer的使用
- apollo 配置中心_分布式配置中心之Apollo
- 为什么要用Mybatis框架---Mybatis学习笔记(一)
- C++ dlopen mini HOWTO 一篇非常好的介绍C++ dlopen linux/mac os x 编程的资料
- JAVA复习(CharSequence接口、RunTime类、System类、object类中的finalize())
- AE鱼眼镜头畸变扭曲修复插件RevisionFX RELens for Mac
- 深度学习与TensorFlow:VGG论文复现
- docker命令每次需要sudo操作解决方案
- 演示:取证分析IPV6组播地址的构成原理
- Mysql删除数据报外键约束解决方法
- Eclipse官方汉化Zip包下载地址及安装方式
- [BJ2006] 狼抓兔子
- 图像处理中的数学原理详解
- Android实现思维导图功能,Android打造思维导图
- 中国四大资产管理公司 ACM
- numpy手写NLP模型(四)———— RNN
- centos服务器无法上网
- vscode 脑图插件mindmap
- 2012第27周移动APP推荐
热门文章
- 前端学习(2312):react之路由基础
- “约见”面试官系列之常见面试题第二十六篇之vue-router的hash和history(建议收藏)
- 前端学习(1885)vue之电商管理系统电商系统之首页路由的重定向
- 24.JSP 客户端请求
- 玩转oracle 11g(18):数据库相关日志文件位置
- linux mysql5.6.27源码安装和错误解决
- lwip+freeRTOS 无法ping通的解决方案
- Linux: 两个USB摄像头的数据采集问题
- linux内核多队列,Linux Kernel 中 Workqueue 使用系统默认队列和创建队列的方法
- RFC函数的初步使用-同步