Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Allow configuration of rebalance retry strategy on HighLevelConsumer #542

Merged
merged 2 commits into from
Jan 3, 2017

Conversation

cesaraugustogarcia
Copy link
Contributor

@cesaraugustogarcia cesaraugustogarcia commented Dec 23, 2016

  • This proposal is to address HighLevelConsumer leaving partitions without owner on multiple consumers startup #541 (HighLevelConsumer leaving partitions without owner on multiple consumers startup). After doing several tests and observations, I believe this (bad) resulting scenario may be caused by race conditions of several consumers trying to consume same partitions and library not handling correctly the retries. Therefore the idea is to allow the rebalance retries to be configurable for each scenario. Particularly, I got good results when setting each consumer with a different (randomised) cycle of retry, to avoid collisions on attempts.
  • The change is not breaking at all.
  • Also is included a bump on retry library (used on rebalancing action) to version 0.7.0 which introduces a couple of bugfixes. The most important fix introduced is Should set _operationTimeout timeout *before* invoking _fn() tim-kos/node-retry#12

Copy link
Collaborator

@hyperlink hyperlink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR.

@@ -23,7 +23,7 @@
"nested-error-stacks": "^2.0.0",
"node-zookeeper-client": "~0.2.2",
"optional": "^0.1.3",
"retry": "~0.6.1",
"retry": "~0.7.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to the latest and use a ^ instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolutely. I just bumped to 0.7.0 trying to be conservative. But I have just checked the latest version, 0.10.0 and has no breaking changes.
So updating my fork changing the commit that has that bump on the retry dependency.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright @hyperlink version changed.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants