I'm developing a system (in java) with at-least-once in-order delivery guarantees. I have a transactional outbox on the producer size, and each message I send has a monotonically increasing counter, so consumers must remember the largest counter value they've seen to discard duplicates. But to make this system work, I need to have a guarantee that if I enqueued a batch of messages (let's say with counter=n..n+10) to the producer (invoked KafkaProducer#send for them in the correct order), they will be sent without gaps, so if a message with counter=n+5 cannot be sent and is discarded from the producer's queue (due to being expired for example), the producer won't send messages with counter=n+6..n+10, so I can retry sending the whole batch or at least messages with counter=n+5..n+10 later from my code.
Simple idempotent producer won't give me that (I see gaps due to messages' expirations), so I decided to use transactions.
If the transaction succeeds, all messages are delivered in the correct order. But if the transaction is aborted (due to message expiration for example), do I have a guarantee that there will be no gaps, so consumers don't need to use READ_COMMITTED isolation level, as they will always see messages coming in order and can safely process uncommitted messages and don't have to wait until the transaction is closed (committed or aborted)?
I don't see such behavior (gaps) when playing with transactions locally (with a toxiproxy between producer and Kafka to emulate network issues), but maybe it is just an implementation detail of the current producer implementation (https://mvnrepository.com/artifact/org.apache.kafka/kafka-clients/3.1.2), so it can change with producer library or kafka version (2.1.0) update