Produce json message with headers via kafka CLI

Question

Try to produce simple message via kafka kafka-console-producer.sh

Message:

{"status": "success", "properties": []}

Headers:

HeaderKey1:HeaderValue1
HeaderKey2:HeaderValue2

It’s not clear from the CLI API how this can be done

Source message.txt file:

HeaderKey1:HeaderValue1
HeaderKey2:HeaderValue2

{"status": "success", "properties": []}

cat message.txt | \
  docker exec -i kafka kafka-console-producer.sh \
  --bootstrap-server localhost:9092 \
  --topic test_topic \
  --property parse.key=true \
  --property key.separator=\n \
  --property headers.delimiter=:

Expect that the headers will go into the headers, and the json will go into the value

Answer 1

I think you are missing the --property parse.headers=true since the default is false. Also I think you need to rewrite your message like this:

h1:v1,h2:v2;{"status": "success", "properties": []}

And produce with this:

--property parse.headers=true  --property headers.delimiter=';'

This will work since the defaults are: headers.key.separator=: and headers.separator=,

Headers:
            Key: h1   Value: v1
            Key: h2   Value: v2
Partition:   2
Offset:      20
Timestamp:   2024-03-05 21:04:07.134 +0000 GMT
{
 "properties": [],
 "status": "success"
}