Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[fix] Key_Shared mode consumption latency when low traffic #23340

Merged
merged 4 commits into from
Sep 23, 2024

Conversation

poorbarcode
Copy link
Contributor

@poorbarcode poorbarcode commented Sep 23, 2024

Motivation

Without #23226 and #23284, the consumption latency is low, but the consumption latency increases to 1s when the traffic is low.

Consumption latency after an upgrade(3 topics).

histogram_quantile(0.98, sum-by (le, topic) (rate (Message_duration_eventstate_ inbound_latencymin_bucket {kubernetes_namespace-"$(env)"} [$__rate_interval])))

Screenshot 2024-09-24 at 00 27 11

Root cause

Screenshot 2024-09-24 at 02 30 27

Modifications

Change the feature's introduced by #23226 default values to 1ms initial / 10ms max.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

Copy link

@poorbarcode Please add the following content to your PR description and select a checkbox:

- [ ] `doc` <!-- Your PR contains doc changes -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update later -->
- [ ] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->

@github-actions github-actions bot added doc-not-needed Your PR changes do not impact docs and removed doc-label-missing labels Sep 23, 2024
Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous solution was burning CPU in a tight loop. Instead of disabling this, the values can be made 100x smaller.

Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with values 1ms initial / 10ms max

@lhotari lhotari changed the title [fix] Key_Shared mode comsumption latency when low traffic [fix] Key_Shared mode consumption latency when low traffic Sep 23, 2024
@codelipenghui codelipenghui added this to the 4.0.0 milestone Sep 23, 2024
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.52%. Comparing base (bbc6224) to head (513498e).
Report is 598 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23340      +/-   ##
============================================
+ Coverage     73.57%   74.52%   +0.95%     
- Complexity    32624    34431    +1807     
============================================
  Files          1877     1934      +57     
  Lines        139502   144981    +5479     
  Branches      15299    15839     +540     
============================================
+ Hits         102638   108049    +5411     
+ Misses        28908    28658     -250     
- Partials       7956     8274     +318     
Flag Coverage Δ
inttests 27.85% <100.00%> (+3.27%) ⬆️
systests 24.55% <100.00%> (+0.23%) ⬆️
unittests 73.88% <100.00%> (+1.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...org/apache/pulsar/broker/ServiceConfiguration.java 98.96% <100.00%> (-0.43%) ⬇️
...sistent/PersistentDispatcherMultipleConsumers.java 75.29% <100.00%> (+0.96%) ⬆️

... and 601 files with indirect coverage changes

@lhotari lhotari merged commit 4ce0c75 into apache:master Sep 23, 2024
55 checks passed
lhotari pushed a commit that referenced this pull request Sep 23, 2024
Co-authored-by: Lari Hotari <lhotari@apache.org>
(cherry picked from commit 4ce0c75)
lhotari pushed a commit that referenced this pull request Sep 23, 2024
Co-authored-by: Lari Hotari <lhotari@apache.org>
(cherry picked from commit 4ce0c75)
lhotari added a commit that referenced this pull request Sep 24, 2024
lhotari added a commit that referenced this pull request Sep 24, 2024
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
)

Co-authored-by: Lari Hotari <lhotari@apache.org>
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants