Missing results after reducing the visualization size

Question

I would like to count the same log messages in Kibana. With the Size set to 200, it turns out that there are two results that happened twice

But, if I lower the Size to 5, I don't see those two:

It should show me top 5 rows, ordered by count. I expected something like this:

| LogMessage | Count |
|------------|-------|
| xx         | 2     |
| yy         | 2     |
| zz         | 1     |
| qq         | 1     |
| ww         | 1     |

What am I missing?

Answer 1

The issue is the little warning about Analyzed Field. You should use a keyword field.

With analyzed fields, the analyzer breaks down the original string during indexing into sub-strings to facilitate search use cases (handling things like word boundaries, punctuation, case insensitivity, declination, etc)

A keyword field is just a simple string.

What's probably happening is that you have data like

| LogMessage | Count |
|------------|-------|
| a          |   1   |
| b          |   1   |
| c x        |   1   |
| d x        |   1   |

With an analyzed field, if you have a terms agg of size 2 you might (depending on the sort order) get a and b

With a larger terms agg, the top sub-string will be x

This is a simplified example, but I hope it gets the issue across.

The Terms Aggregation docs have a good section about how to avoid/solve this issue.