Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[improve] [broker] Check max producers/consumers limitation first before other ops to save resources #23074

Merged
merged 7 commits into from
Jul 29, 2024

Conversation

poorbarcode
Copy link
Contributor

Motivation

Background

  • Pulsar checks compatible of schemas when adding producers/consumers
  • After checking schemas, it checks the max producers/consumers count.

Issue

  • If the clients have already reached max limitation of producers/consumers count, the client will try to register producers/consumers continuously.
  • It checks schemas, then checks the max producers/consumers count. This will waste many CPU resources to check the compatibility of schemas.

Modifications

  • Check max producers/consumers limitation first before other ops to save resources

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

@poorbarcode poorbarcode self-assigned this Jul 25, 2024
@poorbarcode poorbarcode added this to the 3.4.0 milestone Jul 25, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Jul 25, 2024
@poorbarcode poorbarcode requested a review from gaoran10 July 26, 2024 00:23
Copy link
Member

@crossoverJie crossoverJie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, LGTM.

@poorbarcode poorbarcode requested a review from gaoran10 July 26, 2024 15:21
@codecov-commenter
Copy link

codecov-commenter commented Jul 29, 2024

Codecov Report

Attention: Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.

Project coverage is 73.45%. Comparing base (bbc6224) to head (a20ff14).
Report is 603 commits behind head on master.

Files with missing lines Patch % Lines
...rg/apache/pulsar/broker/service/AbstractTopic.java 80.00% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23074      +/-   ##
============================================
- Coverage     73.57%   73.45%   -0.13%     
- Complexity    32624    33212     +588     
============================================
  Files          1877     1917      +40     
  Lines        139502   144077    +4575     
  Branches      15299    15741     +442     
============================================
+ Hits         102638   105828    +3190     
- Misses        28908    30133    +1225     
- Partials       7956     8116     +160     
Flag Coverage Δ
inttests 27.55% <28.57%> (+2.97%) ⬆️
systests 24.76% <21.42%> (+0.43%) ⬆️
unittests 72.50% <92.85%> (-0.35%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...va/org/apache/pulsar/broker/service/ServerCnx.java 72.28% <100.00%> (+0.13%) ⬆️
...rg/apache/pulsar/broker/service/AbstractTopic.java 88.30% <80.00%> (+0.31%) ⬆️

... and 509 files with indirect coverage changes

@poorbarcode poorbarcode merged commit 679a3d4 into apache:master Jul 29, 2024
51 checks passed
poorbarcode added a commit that referenced this pull request Jul 29, 2024
…ore other ops to save resources (#23074)

(cherry picked from commit 679a3d4)
poorbarcode added a commit that referenced this pull request Jul 29, 2024
…ore other ops to save resources (#23074)

(cherry picked from commit 679a3d4)
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 30, 2024
…ore other ops to save resources (apache#23074)

(cherry picked from commit 679a3d4)
(cherry picked from commit f08a34c)
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jul 30, 2024
…ore other ops to save resources (apache#23074)

(cherry picked from commit 679a3d4)
(cherry picked from commit f08a34c)
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants