Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

domain: batch handle the privilege reload event #59934

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

tiancaiamao
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #59400

Problem Summary:

After a NotifyPrivilegeUpdate(), the notification from etcd is laggy.

When we're in a batch of privilege modification operations, the laggy event cause that even when the workload is done,
tidb still have a long period of time handling the laggy events.
This use 100% cpu (1 core) even there are no other events.

What changed and how does it work?

Wait for a while to get a batch of the laggy events, handle them together.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)

Before:
image

After:
image

As you can see, after the workload finish, the CPU drop to 0 immediately with this commit.

  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added release-note-none Denotes a PR that doesn't merit a release note. do-not-merge/needs-triage-completed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 6, 2025
Copy link

tiprow bot commented Mar 6, 2025

Hi @tiancaiamao. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

codecov bot commented Mar 6, 2025

Codecov Report

Attention: Patch coverage is 36.00000% with 16 lines in your changes missing coverage. Please review.

Project coverage is 74.8454%. Comparing base (e6b4d95) to head (cfee950).
Report is 4 commits behind head on master.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #59934        +/-   ##
================================================
+ Coverage   72.9252%   74.8454%   +1.9202%     
================================================
  Files          1701       1748        +47     
  Lines        469722     478399      +8677     
================================================
+ Hits         342546     358060     +15514     
+ Misses       106059      97756      -8303     
- Partials      21117      22583      +1466     
Flag Coverage Δ
integration 48.8213% <36.0000%> (?)
unit 72.2377% <36.0000%> (+0.1012%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 52.6910% <ø> (ø)
parser ∅ <ø> (∅)
br 61.4022% <ø> (+16.5718%) ⬆️
🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

func ignoreBatchData(ch clientv3.WatchChan) {
ticker := time.NewTicker(5 * time.Millisecond)
defer ticker.Stop()
for i := 0; i < 1024; i++ {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does 1024 here means maxBatchSize? Maybe use a const is more reasonable.

Copy link

ti-chi-bot bot commented Mar 6, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Defined2014
Once this PR has been reviewed and has the lgtm label, please assign gmhdbjd for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

ti-chi-bot bot commented Mar 6, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-03-06 03:42:19.67105816 +0000 UTC m=+499452.799977903: ☑️ agreed by Defined2014.

@ti-chi-bot ti-chi-bot bot added the needs-1-more-lgtm Indicates a PR needs 1 more LGTM. label Mar 6, 2025
@tiancaiamao
Copy link
Contributor Author

/retest

Copy link

tiprow bot commented Mar 6, 2025

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tiancaiamao
Copy link
Contributor Author

/retest

Copy link

tiprow bot commented Mar 6, 2025

@tiancaiamao: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
needs-1-more-lgtm Indicates a PR needs 1 more LGTM. release-note-none Denotes a PR that doesn't merit a release note. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

The load privilege event is laggy after the workload gone
2 participants