Skip to content
This repository was archived by the owner on Mar 5, 2025. It is now read-only.

Ysc pluggable scheduler #4

Merged
merged 29 commits into from
Feb 20, 2025
Merged

Ysc pluggable scheduler #4

merged 29 commits into from
Feb 20, 2025

Conversation

yannicks1
Copy link
Collaborator

@yannicks1 yannicks1 commented Feb 11, 2025 •
edited
Loading

Adding spyre specific scheduler

This PR integrates the (spyre specific) scheduler class to replace the upstream scheduler.
it goes along with this ->PR<- in the IBM/vllm repository

Changes:

  • added vllm_spyre/core/scheduler.py: file containing the spyre specific Scheduler class
  • overwriting default scheduler class VllmConfig.scheduler_config.worker_cls in vllm_spyre/platform.py
  • vllm_spyre/envs.py: adding spyre scheduler related env variables.
  • moving spyre_warmup_shapesand code to "load warmup shapes and sort by "speed" shapes" from vllm/config.py to vllm-spyre/platform.py
  • unifying detection of vllm version V1 (not implemented yet)
  • formatting changes

sducouedic and others added 17 commits January 28, 2025 16:37
@sducouedic
setup commit
8b442b1 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
add spyre examples from vllm
83d70e2 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
add format and linting checking (commented some import errors to be f…
f87cb8d 
…ixed)

Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
executing code with moved platform.py, worker, model_executor. Need c…
9297fcd 
…leanup

Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
rename plugin from 'vllm-spyre' to 'spyre'
bad2a6e 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
adding tests
76d556b 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
validate quantization through Platform base class method (rather than…
fafab25 
… validating manually) by setting attribute 'supported_quantization'

Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
set Spyre Platform as OOT (out-of-tree) and remove PlatformEnum.SPYRE…
74b07ab 
… in vllm

Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
move spyre-related environments variable from vllm to plugin
7405bb7 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
modifications to be able to directly use cpu as device when doing inf…
21e6d40 
…erence with Spyre

Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
working Dockerfile.spyre
192361b 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@sducouedic
simple instructions to install vllm-spyre
e1fc17a 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@yannicks1
format check
524be68 
Signed-off-by: Yannick Schnider <Yannick.Schnider1@ibm.com>
@sducouedic
IBM/vllm plugin branch merge conflicts resolving part 1: tests and build
70bfe65 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
@yannicks1
integrating spyre specific scheduler in plugin
4a05269 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
formatting
1bd010d 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
moving default scheduler def to config.py
616f708 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
This was referenced Feb 11, 2025
Merged
Closed
@yannicks1
moving spyre specific env vars and scheduler configs to plugin
411a4bb 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
unifying vllm V1 check
d9f423c 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@sducouedic sducouedic force-pushed the sop-dev branch 2 times, most recently from 39518b5 to 900a386 Compare February 11, 2025 15:48
@yannicks1 yannicks1 force-pushed the ysc-pluggable-scheduler branch from fbbcc52 to d9f423c Compare February 11, 2025 16:17
@yannicks1 yannicks1 marked this pull request as ready for review February 11, 2025 16:19
@sducouedic sducouedic force-pushed the sop-dev branch 2 times, most recently from 0547dd5 to a5d8fab Compare February 11, 2025 20:00
@yannicks1
moving spyre_warmup_shapes form scheduler config to platform
9d7eb06 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
importing reused classes in vllm-spyre/core/scheduler.py from vllm.co…
1a56df6 
…re.scheduler

Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@sducouedic sducouedic force-pushed the sop-dev branch 2 times, most recently from c3f76d0 to 9209da2 Compare February 13, 2025 13:09
sducouedic
sducouedic previously approved these changes Feb 13, 2025
View reviewed changes
@sducouedic
Copy link
Member

sducouedic commented Feb 13, 2025 •
edited
Loading

We will need to update the Dockerfile.spyre to target upstream-vllm rather than IBM-vllm on this branch before merging (to main rather than sop-dev), after your other PR has been accepted.

@sducouedic sducouedic dismissed their stale review February 13, 2025 20:02

Need to wait for upstream scheduler plugin PR merged

@yannicks1
moving scheduler_cls from parallel_config to scheduler_config
3ef13ea 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
Base automatically changed from sop-dev to main February 18, 2025 13:56
yannicks1 and others added 7 commits February 18, 2025 22:59
@yannicks1
Merge branch 'main' into ysc-pluggable-scheduler
33e0bdc
@yannicks1
removing workflow folder (workflows still here)
fb59447 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
corrected typing for variable length tuple
7dcdf7e 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
removing last empty line
98f9983 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
yapf reformatting
cd3dc65 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@yannicks1
isort reformatting
05ce448 
Signed-off-by: Yannick Schnider <yannick.schnider1@ibm.com>
@sducouedic
yapf formatting: ignore lines that conflict with isort
d8ada06 
Signed-off-by: Sophie du Couédic <sop@zurich.ibm.com>
tdoublep
tdoublep approved these changes Feb 20, 2025
View reviewed changes
Copy link
Member

@tdoublep tdoublep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@tdoublep tdoublep merged commit df4029c into main Feb 20, 2025
9 checks passed
@tdoublep tdoublep deleted the ysc-pluggable-scheduler branch February 20, 2025 10:38
# for free to subscribe to this conversation on GitHub. Already have an account? #.
Reviewers

@maxdebayser maxdebayser maxdebayser left review comments

@tdoublep tdoublep tdoublep approved these changes

@sducouedic sducouedic sducouedic left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yannicks1 @sducouedic @maxdebayser @tdoublep