Well,不要刷屏了

elasticsearch慢查询问题

Elasticsearch | 作者 miqi1992 | 发布于2018年09月07日 | 阅读数:2472


{ "from": 0, "size": 0, "query": { "bool": { "filter": [ { "bool": { "must": [ { "bool": { "must": [ { "range": { "logTime": { "from": null, "to": null, "include_lower": true, "include_upper": true, "boost": 1 } } }, { "range": { "logTime": { "from": null, "to": null, "include_lower": true, "include_upper": true, "boost": 1 } } }, { "bool": { "must_not": [ { "match_phrase": { "errorLevel": { "query": "D", "slop": 0, "boost": 1 } } } ], "disable_coord": false, "adjust_pure_negative": true, "boost": 1 } }, { "match_phrase": { "pageName": { "query": "登录", "slop": 0, "boost": 1 } } }, { "bool": { "should": [ { "match_phrase": { "mobileAppId": { "query": "29fde88202134aaa9513d5cd77872c96", "slop": 0, "boost": 1 } } }, { "match_phrase": { "mobileAppId": { "query": "1f19414c6ad5420a97fad8e8de2a12ac", "slop": 0, "boost": 1 } } }, { "match_phrase": { "mobileAppId": { "query": "e220c7a823e54f6aada37d97465475d3", "slop": 0, "boost": 1 } } } ], "disable_coord": false, "adjust_pure_negative": true, "boost": 1 } } ], "disable_coord": false, "adjust_pure_negative": true, "boost": 1 } } ], "disable_coord": false, "adjust_pure_negative": true, "boost": 1 } } ], "disable_coord": false, "adjust_pure_negative": true, "boost": 1 } }, "_source": { "includes": [ "COUNT", "COUNT", "COUNT" ], "excludes": [ ] }, "aggregations": { "logTime": { "date_histogram": { "field": "logTime", "format": "yyyy-MM-dd HH:mm:ss", "time_zone": "+08:00", "interval": "5m", "offset": 0, "order": { "_key": "asc" }, "keyed": false, "min_doc_count": 0 }, "aggregations": { "errorTypeNum": { "cardinality": { "field": "errorType" } }, "effectUserNum": { "cardinality": { "field": "deviceToken" } }, "effectCount": { "value_count": { "field": "_index" } } } } } }
 
上面是我的查询语句,明明页面请求是带的时间范围,怎么通过es慢查询日志这里from就为null了?
已邀请:

zqc0512 - andy zhou

赞同来自:

用scroll
 

JackGe

赞同来自:

您的查询语句很奇怪,logTime中from和to为null,并且出现两个相同的range查询,includes中出现3个相同的
COUNT,您能打印请求时的查询语句吗(searchSourceBuilder.toString()),感觉和slowlog里的不同?
 
我试了下from和to为null的查询,就是查询所有数据。您的这个查询语句query部分由短语匹配和范围过滤组成,主要耗时不在query过滤部分,是在aggs部分,先是5分钟一个分桶,然后对每个分桶再进行字段errorType和deviceToken进行cardinality计算,这部分会计算会很耗内存和时间。何况range查询中from和to为null导致range查询失效而对所有数据进行聚合计算,5分钟一个分桶,如果是一天的话288个分桶,再乘以
字段errorType和deviceToken的基数能估算出分桶数。具体查询耗时可以通过GET/_search {"profile":true,"query":{您的查询语句}}进行分析
 
以下部分语句很奇怪
{
  "range": {
    "logTime": {
      "from": null,
      "to": null,
      "include_lower": true,
      "include_upper": true,
      "boost": 1
    }
  }
},
{
  "range": {
    "logTime": {
      "from": null,
      "to": null,
      "include_lower": true,
      "include_upper": true,
      "boost": 1
    }
  }
},

"_source": {
"includes": [
"COUNT",
"COUNT",
"COUNT"
],
"excludes": [

]
},

要回复问题请先登录注册