Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Feature] support event scheduler (part 1) #54260

Merged
merged 10 commits into from
Dec 25, 2024

Conversation

stdpain
Copy link
Contributor

@stdpain stdpain commented Dec 24, 2024

What I'm doing:

Adds tool class for event scheduler

link #54259

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.4
    • 3.3
    • 3.2
    • 3.1
    • 3.0

@stdpain stdpain requested review from a team as code owners December 24, 2024 06:44
@wanpengfei-git wanpengfei-git requested a review from a team December 24, 2024 06:44
@stdpain stdpain linked an issue Dec 24, 2024 that may be closed by this pull request
3 tasks
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch 2 times, most recently from 9ddad58 to 4eb5909 Compare December 24, 2024 07:02
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch 2 times, most recently from 4e53bf9 to edab262 Compare December 24, 2024 07:24
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch from edab262 to cc8f32d Compare December 24, 2024 07:36
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch from 54a4fac to 142ea2a Compare December 24, 2024 08:04
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain changed the title [Feature] support phased scheduler [part 1] [Feature] support event scheduler [part 1] Dec 24, 2024
@stdpain stdpain changed the title [Feature] support event scheduler [part 1] [Feature] support event scheduler (part 1) Dec 24, 2024
Signed-off-by: stdpain <drfeng08@gmail.com>
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch from 3c15229 to 8d6abcf Compare December 24, 2024 12:21
Signed-off-by: stdpain <drfeng08@gmail.com>
Signed-off-by: stdpain <drfeng08@gmail.com>
Signed-off-by: stdpain <drfeng08@gmail.com>
be/src/exec/pipeline/pipeline_driver.cpp Show resolved Hide resolved
be/src/exec/pipeline/pipeline_driver.h Show resolved Hide resolved
be/src/exec/pipeline/schedule/event_scheduler.cpp Outdated Show resolved Hide resolved
be/src/exec/pipeline/schedule/observer.cpp Show resolved Hide resolved
be/src/exec/pipeline/schedule/observer.h Outdated Show resolved Hide resolved
Signed-off-by: stdpain <drfeng08@gmail.com>
@stdpain stdpain force-pushed the cmt_event_based_scheduler branch from d6b4d28 to 2428618 Compare December 25, 2024 04:55
Copy link

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

Copy link

[FE Incremental Coverage Report]

pass : 2 / 2 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/qe/SessionVariable.java 2 2 100.00% []

Copy link

[BE Incremental Coverage Report]

pass : 156 / 169 (92.31%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/exec/pipeline/noop_sink_operator.h 0 1 00.00% [58]
🔵 be/src/exec/pipeline/empty_set_operator.h 0 1 00.00% [44]
🔵 be/src/exec/pipeline/operator.h 1 2 50.00% [417]
🔵 be/src/exec/pipeline/pipeline_driver_queue.cpp 5 6 83.33% [70]
🔵 be/src/exec/pipeline/schedule/event_scheduler.cpp 38 42 90.48% [61, 62, 80, 81]
🔵 be/src/exec/pipeline/schedule/pipeline_timer.cpp 31 34 91.18% [29, 51, 64]
🔵 be/src/exec/pipeline/schedule/observer.cpp 15 16 93.75% [78]
🔵 be/src/exec/pipeline/pipeline_driver.h 24 25 96.00% [488]
🔵 be/src/exec/pipeline/pipeline_driver_executor.cpp 1 1 100.00% []
🔵 be/src/exec/pipeline/fragment_context.cpp 5 5 100.00% []
🔵 be/src/exec/pipeline/pipeline_driver.cpp 4 4 100.00% []
🔵 be/src/exec/pipeline/fragment_context.h 2 2 100.00% []
🔵 be/src/exec/pipeline/schedule/event_scheduler.h 1 1 100.00% []
🔵 be/src/exec/pipeline/schedule/utils.h 8 8 100.00% []
🔵 be/src/exec/pipeline/schedule/pipeline_timer.h 8 8 100.00% []
🔵 be/src/exec/pipeline/schedule/observer.h 13 13 100.00% []

@@ -174,6 +175,10 @@ class FragmentContext {
// acquire runtime filter from cache
void acquire_runtime_filters();

bool enable_event_scheduler() const { return event_scheduler() != nullptr; }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: shall it be event_scheduler_enabled? enable_event_scheduler reads like to perform an action, turn on the event_scheduler, not query the on/off status.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

our session variables style is enable_xxx_feature

@stdpain stdpain merged commit 7bc3961 into StarRocks:main Dec 25, 2024
55 of 58 checks passed
maggie-zhu pushed a commit to maggie-zhu/starrocks that referenced this pull request Jan 6, 2025
Signed-off-by: stdpain <drfeng08@gmail.com>
# for free to join this conversation on GitHub. Already have an account? # to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Supports pipeline event based scheduling.
5 participants