I am trying to create a number of aggregations, where each contains only results with the field I filter on. I am able to do this, but the problem is that it also returns empty buckets and I am wanting to foreach over them - I do not want an action to occur on empty results.
I currently have something like this:
"aggs" : {
"colour_agg" : {
"filters": {
"filters": [
{ "term" : { "COLOUR" : "Red" }},
{ "term" : { "COLOUR" : "Blue" }},
{ "term" : { "COLOUR" : "Yellow" }}
]
},
"aggs" : {
"colour_agg" : {
"terms" : {
"field" : "COLOUR",
"min_doc_count" : 150
},
"aggs" : {
"hex_agg" : {
"terms" : {
"field" : "HEX"
}
}
}
which then returns this:
"colour_agg" : {
"buckets" : [
{
"doc_count" : 116,
"colour_agg" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [ ]
}
},
{
"doc_count" : 256,
"colour_agg" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"doc_count" : 256,
"key" : "Blue",
"hex_agg" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [
{
"doc_count" : 256,
"key" : "0000ff"
}
]
}
}
]
}
},
{
"doc_count" : 0,
"event_agg" : {
"doc_count_error_upper_bound" : 0,
"sum_other_doc_count" : 0,
"buckets" : [ ]
}
}
]
}
}
This means that when I loop over the colour_agg buckets, my action fires 3 times. In this scenario, I would want it to fire only once. I only want there to be a bucket created when there are 150 results found.
How can I remove the empty buckets from the colour_agg result? I have tried using min_doc_count at the filter level but this doesn't work, and by the time it is applied lower down the tip-level filter has already created empty buckets. I do not seem to be able to reverse this order, and have seen that post_filter (if a workable solution here) brings a performance hit with it.
Thanks.
Full watcher:
{
"trigger": {
"schedule": {
"interval": "30m"
}
},
"input": {
"search": {
"request": {
"search_type": "query_then_fetch",
"indices": "ABC",
"body": {
"size": 0,
"query": {
"bool": {
"must": {
"match": {
"COMPONENT": "69"
}
},
"filter": {
"range": {
"@timestamp": {
"gte": "now-24h"
}
}
}
}
},
"aggs" : {
"colours" : {
"filters": {
"filters": [
{ "term" : { "COLOUR" : "Red" }},
{ "term" : { "COLOUR" : "Blue" }},
{ "term" : { "COLOUR" : "Yellow" }}
]
},
"aggs" : {
"colour_agg" : {
"terms" : {
"field" : "COLOUR",
"min_doc_count" : 150
},
"aggs" : {
"hex_agg" : {
"terms" : {
"field" : "HEX"
}
}
}
}
}
}
}
}
}
}
},
"condition": {
"compare": {
"ctx.payload.hits.total": {
"gt": 0
}
}
},
"actions": {
"teams_message" : {
"foreach": "ctx.payload.aggregations.colours.buckets",
...
}
}
}