Is there a limit on the number of filters that can be passed to DynamoDB query

Question

I’d like to search for something like

Field “Id” has a value in [Long list of IDs]

This long list of Ids can hit over 1000 Ids.

should I expect a problem with that? Is there a limit on how long the query can be?

I am looking at cloudsearch and it seems to have a limit of 1024 clauses and wondering if it should just be done from DynamoDB if there are no limits on it.

At that point, I guess I should also ask if Elastic search/Open search has such limits,

Answer 1

You can review the various DynamoDB limits at https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/ServiceQuotas.html. Here's two that will impact you:

The maximum length of any expression string is 4 KB.

The maximum number of operands for the IN comparator is 100.

Answer 2

Elasticsearch has a limit on max clause a query can have as explained in the search settings, but if you are using it only to filter the data, as you mentioned in your question, than you can simply use the terms query where you can send a long list of ids to filter on, this is also advised by Elasticsearch in the same document search settings.

Answer 3

Field “Id” has a value in [Long list of IDs]
This long list of Ids can hit over 1000 Ids.

Is Id your primary key? If so, no because in dynamo you can do batchGet operation on passing the Ids and for 1000 you will have to do 10 concurrent/sequential calls to dynamo.

if it's not a primary key, i.e. a secondary index then you will have to do 1000 concurrent query operation to check the presence of the key.