Cannot read from beginning of topic using segmentio/kafka-go

Question

I am unable to read from the beginning of a topic using kafka-go. Here's how I create the reader:

    reader := kafka.NewReader(kafka.ReaderConfig{
        Brokers: []string{"127.0.0.1:9092"},
        GroupID: "my-group-id",
        Topic:   "my-topic",
    })
    reader.SetOffset(0)

    for {
        message, err := reader.ReadMessage(context.Background())

        if err != nil {
            log.Errorln("ReadMessage failed:", err)
        }

        log.Info("Received: ", string(message.Value))
        ...

I've tried passing kafka.FirstOffset and kafka.LastOffset as well but all result in the reading from the latest offset when I restart my app. I get the same result if I set the StartOffset field to 0 in the kafka.ReaderConfig object.

Answer 1

In order to specify the start offset, I had to specify the partition:

reader := kafka.NewReader(kafka.ReaderConfig{
    Brokers: []string{"127.0.0.1:9092"},
    Topic:   "my-topic",
    Partition: 0,
    StartOffset: 0,
})

You cannot specify both GroupID and Partition to set the StartOffset.