Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[fix] [broker] Enabling batch causes negative unackedMessages due to ack and delivery concurrency #22090

Merged
merged 3 commits into from
Feb 26, 2024

Conversation

poorbarcode
Copy link
Contributor

Motivation

Context:

  • After a consumer closes, it may leave an in-progress acknowledge task.

Issue:

  • There are two consumers: c1 and c2
  • c1 received a batch message: ledger: L1, entry: E1, batch-size: 100.
  • c1 was closed, after leaving an in-progress task to acknowledge {L1, E1, batch_index: 1}
  • Pulsar read {L1, E1} and tries to delivery to c2.
  • The acknowledgement {L1, E1, batch_index: 1} is finished.
  • c2 gets 100 messages, but unackedMessages of c2 is 98.

We can always get the warn logs like below:

07:01:11.037 [pulsar-io-4-17] WARN  org.apache.pulsar.broker.service.Consumer - unackedMsgs is : -1,

Modifications

Fix the bug.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository: x

@poorbarcode poorbarcode added this to the 3.3.0 milestone Feb 21, 2024
@poorbarcode poorbarcode self-assigned this Feb 21, 2024
@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Feb 21, 2024
@Technoboy-
Copy link
Contributor

Some tests are failed due to this fix

@poorbarcode
Copy link
Contributor Author

Some tests are failed due to this fix

passes

@poorbarcode poorbarcode merged commit 1b1cfb5 into apache:master Feb 26, 2024
50 checks passed
Technoboy- pushed a commit that referenced this pull request Feb 26, 2024
Technoboy- pushed a commit that referenced this pull request Feb 27, 2024
heesung-sn pushed a commit that referenced this pull request Feb 27, 2024
…ack and delivery concurrency (#22090)

(cherry picked from commit 1b1cfb5)
poorbarcode added a commit that referenced this pull request Feb 28, 2024
…ack and delivery concurrency (#22090)

(cherry picked from commit 1b1cfb5)
(cherry picked from commit b961c49)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 1, 2024
…ack and delivery concurrency (apache#22090)

(cherry picked from commit aa3f53b)
mukesh-ctds pushed a commit to datastax/pulsar that referenced this pull request Mar 6, 2024
…ack and delivery concurrency (apache#22090)

(cherry picked from commit aa3f53b)
hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants