Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

How to set consumer option "auto.offset.reset" ? #342

Closed
nobusue opened this issue Mar 28, 2016 · 8 comments
Closed

How to set consumer option "auto.offset.reset" ? #342

nobusue opened this issue Mar 28, 2016 · 8 comments

Comments

@nobusue
Copy link

nobusue commented Mar 28, 2016

I'd like to consume from latest message when I don't have committed offset (ex: new consumer group).
At v0.3.2, HighLevelConsumer consumes from oldest message, but it isn't expected.

@haio
Copy link
Member

haio commented Mar 31, 2016

You can set fromOffset true and give the offset you wish to consume in payload, example:

consumer = new HighLevelConsumer(
        client,
        [
            { topic: 't', offset: 1000 }
        ],
        { fromOffset: true }
    );

@nobusue
Copy link
Author

nobusue commented Apr 1, 2016

Thanks haio, but it doesn't match my needs.
I just intend to start consume latest message for "fresh consumer" (= there is no committed offset entry in zookeeper). In other language like Java, Kafka client API supports to set option map, like "auto.offset.reset=latest".
Is there any way to set Kafka client API option map directly?

@haio
Copy link
Member

haio commented Apr 1, 2016

This feature is not supported at the moment.

@CharlesWall
Copy link
Contributor

I'll have a PR coming for picking up from the latest offset.

@ghost
Copy link

ghost commented Feb 9, 2017

@CharlesWall any option like auto.offset.reset=latest , sometime soon for 0.8.* version of kafka for node client.

@hyperlink
Copy link
Collaborator

@shubSK I wouldn't hold my breath waiting for that.

@CharlesWall
Copy link
Contributor

I haven't looked at any of this stuff since May. I learned how this module worked to write the fix I needed. @shubSK You should consider doing the same.

@hyperlink
Copy link
Collaborator

The solution is to use ConsumerGroup which supports both earliest and latest. I don't think it's worth updating the HLC since it has been deprecated by Kafka project.

# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

No branches or pull requests

4 participants