Elasticsearch Trying to create too many scroll contexts. Must be less than or equal to: [500]

在做数据导出的时候，因为用到了滚动。因为每次调用都创建了 scroll，而且我没有手动删除，而es 默认的 scroll 是500个。到达500个没有清除的手，就会报这个错误。错误如下

{"error":{"root_cause":[{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."},{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"collect_data_store_customer_flow_fact_201910","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}},{"shard":0,"index":"collect_data_store_customer_flow_fact_201911","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}}]},"status":500} at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:920) at org.elasticsearch.client.RestClient.performRequest(RestClient.java:227) at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1433) ... 62 common frames omitted Caused by: org.elasticsearch.client.ResponseException: method [POST], host [http://172.18.32.181:9200], URI [/collect_data_store_customer_flow_fact*/_search?typed_keys=true&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&scroll=60000ms&search_type=query_then_fetch&batched_reduce_size=512&ccs_minimize_roundtrips=true], status line [HTTP/1.1 500 Internal Server Error] {"error":{"root_cause":[{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."},{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"collect_data_store_customer_flow_fact_201910","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}},{"shard":0,"index":"collect_data_store_customer_flow_fact_201911","node":"4oJe_iuMS4eEUxOEVRt6hQ","reason":{"type":"exception","reason":"Trying to create too many scroll contexts. Must be less than or equal to: [500]. This limit can be set by changing the [search.max_open_scroll_context] setting."}}]},"status":500} at org.elasticsearch.client.RestClient$1.completed(RestClient.java:540) at org.elasticsearch.client.RestClient$1.completed(RestClient.java:529) at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39) at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)

# # 解决方式

1. 在 kibana 上边执行 DELETE _search/scroll/_all

2. 也可以通过下边的方式来清楚。

curl -XDELETE localhost:9200/_search/scroll/_all

# # 总结

如果经常用到 scroll 的话，还是要手动清除的。也可以在程序中清除，更好一点。

出现这样的问题，就好好检查一下代码逻辑吧，是不是逻辑哪里有问题，在频繁的创建scroll

Elasticsearch Trying to create too many scroll contexts. Must be less than or equal to: [500]相关推荐

elasticsearch index、create和update的源码分析
https://segmentfault.com/a/1190000011272749 社区里面有人问了如下一个问题: 执行 bulk 索引文档的时候,用 index 或者 create 类型并且自定 ...
Elasticsearch分页查询Fromamp;Size VS scroll
对于ES来说,按照一般的查询流程来说,如果我想查询数据: 1 客户端请求发给某个节点 2 节点转发给个个分片,查询每个分片上的前10条 3 结果返回给节点,整合数据,提取前10条 4 返回给请求客户端 ...
elasticsearch 大数据场景下使用scroll实现分页查询
es查询大批量数据的"可能方案" 当使用es来请求大批量数据时,通常有三种办法,其一:直接查询获取全量数据:其二:使用setFrom以及setSize解决:其三:使用es自带的sc ...
elasticsearch Trying to create too many buckets
问题:Trying to create too many buckets. Must be less than or equal to: [20000] but was [86391]. This l ...
ElasticSearch（四）：DSL Query
ES中提供了一种强大的检索数据方式,这种检索方式称之为Query DSL(Domain Specified Language) Query DSL是利用Rest API传递JSON格式的请求体(Req ...
Elasticsearch分页查询FromSize vs scroll vs search_after
背景 Elasticsearch 是一个实时的分布式搜索与分析引擎,被广泛用来做全文搜索.结构化搜索.分析.在使用过程中,有一些典型的使用场景,比如分页.遍历等.在使用关系型数据库中,我们被告知要注意 ...
elasticsearch 分页查询实现方案——Top K+归并排序
elasticsearch 分页查询实现方案 1. from+size 实现分页 from表示从第几行开始,size表示查询多少条文档.from默认为0,size默认为10, 注意:size的大小不能 ...
Elasticsearch cat Apis
1._cat列入所有有效命令 GET /_cat返回:有个猫...所以不难想象为啥是cat api =^.^= /_cat/allocation /_cat/shards /_cat/shards/{ ...
Elasticsearch：cat API 介绍及其使用
在 Elasticsearch 中,cat API 仅适用于使用 Kibana 控制台或命令行供人类使用. 它们不适合应用程序使用. 对于应用程序消耗,我们建议使用相应的 JSON API.所有的 c ...
[Elasticsearch]cat API的使用
毕业没多久,由于工作原因,被老大安排学习elasticsearch并用于日志检索以提高线上日志的排查效率.那时候对于很多东西都没有概念,对elasticsearch的理解也不够全面,所以搞出来的东西也 ...

Elasticsearch Trying to create too many scroll contexts. Must be less than or equal to: [500]

Elasticsearch Trying to create too many scroll contexts. Must be less than or equal to: [500]相关推荐

最新文章

热门文章