consumer appears stuck when storing an earlier offset

[nkostoulas]

I recently noticed an issue with storing offsets. Our consumer receives batches of messages using rd_kafka_poll and then concurrently process these and stores offsets. What happened is that our consumer stored the latest offset, say X, but due to a concurrency issue this was followed up by a call to rd_kafka_store_offsets with an earlier offset, say X-5. Since we don't commit directly, it happens every 2 seconds, it seems that the library committed X-5 instead of X (broker scripts showed a lag of 5 when descring consumer group) and the messages from X-5 to X where never returned again when calling rd_kafka_poll.

Is this normal behaviour? I would expect either librdkafka to commit the latest offset or to return the messages again.

The following config is used:

"enable.auto.offset.store": false,
"auto.commit.interval.ms":  2000

For the record this is using confluent-kafka-go. Can do further investigation but wanted to reach out for some first thoughts.