Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Bugfix] Fix a path bug in disaggregated prefill example script. #12121

Merged
merged 2 commits into from
Jan 17, 2025
Merged

[Bugfix] Fix a path bug in disaggregated prefill example script. #12121

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
11134b6
adjust the path
KuntaiDu Jan 16, 2025
4b888cf
add -xe flag to make sure the script crashes when error happens
KuntaiDu Jan 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion examples/online_serving/disaggregated_prefill.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
# We will launch 2 vllm instances (1 for prefill and 1 for decode),
# and then transfer the KV cache between them.

set -xe

echo "🚧🚧 Warning: The usage of disaggregated prefill is experimental and subject to change 🚧🚧"
sleep 1

Expand Down Expand Up @@ -69,7 +71,7 @@ wait_for_server 8200
# instance
# NOTE: the usage of this API is subject to change --- in the future we will
# introduce "vllm connect" to connect between prefill and decode instances
python3 ../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
python3 ../../benchmarks/disagg_benchmarks/disagg_prefill_proxy_server.py &
sleep 1

# serve two example requests
Expand Down
Loading