ElasticSearch problem when I try to save a new product. Error: "Limit of total fields [1000] has been exceeded"

Question

I have a problem when I try to save a product in my collection. When I try to save the product number 271 I obtain this error

Error creating entity in index product_ids: Invalid Elasticsearch response built from a unsuccessful (400) low level call on PUT: /product_ids/_doc/myproduct

Exception: Request failed to execute. Call: Status code 400 from: PUT /product_ids/_doc/myproduct. ServerError: Type: illegal_argument_exception Reason: "Limit of total fields [1000] has been exceeded"

I thought that the problem was due to the fact that the sum of all the fields of my products was greater than 1000 but if I remove a compound field within the product I end up inserting even more than a thousand products in the index. What could be the error?

Answer 1

Elasticsearch by default will give you 1000 field limit for each index for performance purposes. You can increase the field limit using this:

PUT product_ids(your index)/_settings
{
  "index.mapping.total_fields.limit": 2000
}

2000 would be the new field limit.
PS: I understand needing to increase field limit but I'd suggest not to increase it too much because in large scale applications it will start to affect performance. Try to design documents different way like don't use UUIDs as key, or use a list instead of a dict if possible.

Answer 2

The max size of fields is not per document but per index. Get the mapping with GET /product_ids/_mapping and check what you have.

Do you really need that number of fields? May be you could design your documents in another way?