Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

Upstream merge 25 01 13 #358

Merged
merged 116 commits into from
Jan 14, 2025
Merged

Upstream merge 25 01 13 #358

merged 116 commits into from
Jan 14, 2025

Conversation

gshtras
Copy link
Collaborator

@gshtras gshtras commented Jan 13, 2025 •
edited by github-actions bot
Loading

Upstream merge

  • reverting memory profiling changes due to it showing excessive memory usage

jeejeelee and others added 30 commits January 6, 2025 15:22
@jeejeelee
[Bugfix][V1] Fix molmo text-only inputs (vllm-project#11676)
32c9eff 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
@heheda12345
[Kernel] Move attn_type to Attention.__init__() (vllm-project#11690)
e20c92b 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@ywang96 @DarkLight1337
[V1] Extend beyond image modality and support mixed-modality inferenc…
91b361a 
…e with Llava-OneVision (vllm-project#11685)

Signed-off-by: Roger Wang <ywang@roblox.com>
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Bugfix] Fix LLaVA-NeXT feature size precision error (for real) (vllm…
08fb75c 
…-project#11772)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@DarkLight1337
[Model] Future-proof Qwen2-Audio multi-modal processor (vllm-project#…
d0169e1 
…11776)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@ys950902
[XPU] Make pp group initilized for pipeline-parallelism (vllm-project…
d93d2d7 
…#11648)

Signed-off-by: yisheng <yi.sheng@intel.com>
@DarkLight1337
[Doc][3/N] Reorganize Serving section (vllm-project#11766)
8ceffbf 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@jeejeelee @Abatom
[Kernel][LoRA]Punica prefill kernels fusion (vllm-project#11234)
b278557 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Signed-off-by: Abatom <abzhonghua@gmail.com>
Co-authored-by: Zhonghua Deng <abatom@163.com>
@ywang96
[Bugfix] Update attention interface in Whisper (vllm-project#11784)
0f3f3c8 
Signed-off-by: Roger Wang <ywang@roblox.com>
@liangfu
[CI] Fix neuron CI and run offline tests (vllm-project#11779)
898cdf0 
Signed-off-by: Liangfu Chen <liangfc@amazon.com>
@XiaobingSuper
fix init error for MessageQueue when n_local_reader is zero (vllm-pro…
e512f76 
…ject#11768)
@russellb
[Doc] Create a vulnerability management team (vllm-project#9925)
ce1917f 
Signed-off-by: Russell Bryant <rbryant@redhat.com>
@zhouyuan
[CI][CPU] adding build number to docker image name (vllm-project#11788)
1e4ce29 
Signed-off-by: Yuan Zhou <yuan.zhou@intel.com>
@ywang96
[V1][Doc] Update V1 support for LLaVa-NeXT-Video (vllm-project#11798)
8082ad7 
Signed-off-by: Roger Wang <ywang@roblox.com>
@DarkLight1337
[Bugfix] Comprehensively test and fix LLaVA-NeXT feature size calcula…
8f37be3 
…tion (vllm-project#11800)

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao @DarkLight1337
[doc] add doc to explain how to use uv (vllm-project#11773)
869e829 
Signed-off-by: youkaichao <youkaichao@gmail.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@ywang96
[V1] Support audio language models on V1 (vllm-project#11733)
2de197b 
Signed-off-by: Roger Wang <ywang@roblox.com>
@youkaichao
[doc] update how pip can install nightly wheels (vllm-project#11806)
d9fa1c0 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@DarkLight1337
[Doc] Add note to gte-Qwen2 models (vllm-project#11808)
c0efe92 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@youkaichao
[optimization] remove python function call for custom op (vllm-projec…
869579a 
…t#11750)

Signed-off-by: youkaichao <youkaichao@gmail.com>
@jiangjiadi
[Bugfix] update the prefix for qwen2 (vllm-project#11795)
c994223 
Co-authored-by: jiadi.jjd <jiadi.jjd@antgroup.com>
@sroy745
[Doc]Add documentation for using EAGLE in vLLM (vllm-project#11417)
973f5dc 
Signed-off-by: Sourashis Roy <sroy@roblox.com>
@DamonFool
[Bugfix] Significant performance drop on CPUs with --num-scheduler-st…
a4e2b26 
…eps > 1 (vllm-project#11794)
@hmellor
[Doc] Group examples into categories (vllm-project#11782)
5950f55 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@DarkLight1337
[Bugfix] Fix image input for Pixtral-HF (vllm-project#11741)
91445c7 
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
@divakar-amd
[Misc] sort torch profiler table by kernel timing (vllm-project#11813)
4d29e91
@WangErXiao
Remove the duplicate imports of MultiModalKwargs and PlaceholderRange… (
dc71af0 
vllm-project#11824)
@npanpaliya
Fixed docker build for ppc64le (vllm-project#11518)
b640b19 
Signed-off-by: Nishidha Panpaliya <nishidha.panpaliya@partner.ibm.com>
@ilya-lavrenov
[OpenVINO] Fixed Docker.openvino build (vllm-project#11732)
f4923cb 
Signed-off-by: Ilya Lavrenov <ilya.lavrenov@intel.com>
@jeejeelee
[Bugfix] Add checks for LoRA and CPU offload (vllm-project#11810)
f645eb6 
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
ywang96 and others added 27 commits January 12, 2025 06:36
@ywang96
[V1] Avoid sending text prompt to core engine (vllm-project#11963)
b25cfab 
Signed-off-by: Roger Wang <ywang@roblox.com>
@rafvasq
[CI/Build] Add markdown linter (vllm-project#11857)
43f3d9e 
Signed-off-by: Rafael Vasquez <rafvasq21@gmail.com>
@Isotr0py @DarkLight1337
[Model] Initialize support for Deepseek-VL2 models (vllm-project#11578)
f967e51 
Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@Akshat-Tripathi @mosalov
@jeejeelee @Isotr0py
[Hardware][CPU] Multi-LoRA implementation for the CPU backend (vllm-p…
8bddb73 
…roject#11100)

Signed-off-by: Akshat Tripathi <akshat@krai.ai>
Signed-off-by: Oleg Mosalov <oleg@krai.ai>
Signed-off-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Oleg Mosalov <oleg@krai.ai>
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
Co-authored-by: Isotr0py <2037008807@qq.com>
@avshalomman
[Hardware][TPU] workaround fix for MoE on TPU (vllm-project#11764)
263a870
@robertgshaw2-redhat
[V1][Core][1/n] Logging and Metrics (vllm-project#11962)
9597a09 
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
@Isotr0py @DarkLight1337
[Model] Support GGUF models newly added in transformers 4.46.0 (vll…
d14e98d 
…m-project#9685)

Signed-off-by: Isotr0py <2037008807@qq.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
@robertgshaw2-redhat
[V1] [2/n] Logging and Metrics - OutputProcessor Abstraction (vllm-…
619ae26 
…project#11973)

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
@yyccli
[MISC] fix typo in kv transfer send recv test (vllm-project#11983)
f7b3ba8
@liaoyanqing666
[Bug] Fix usage of .transpose() and .view() consecutively. (vllm-…
9dd02d8 
…project#11979)
@llsj14
[CI][Spec Decode] fix: broken test for EAGLE model (vllm-project#11972)
80ea3af 
Signed-off-by: Sungjae Lee <33976427+llsj14@users.noreply.github.com>
@Concurrensee
[Misc] Fix Deepseek V2 fp8 kv-scale remapping (vllm-project#11947)
cf6bbcb 
Signed-off-by: Yida Wu <yidawu@alumni.cmu.edu>
@noemotiovon
[Misc]Minor Changes about Worker (vllm-project#11555)
c3f05b0 
Signed-off-by: Chenguang Li <757486878@qq.com>
@youkaichao
[platform] add ray_device_key (vllm-project#11948)
89ce62a 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@alex-jw-brooks
Fix Max Token ID for Qwen-VL-Chat (vllm-project#11980)
5340a30 
Signed-off-by: Alex-Brooks <Alex.brooks@ibm.com>
@heheda12345
[Kernel] unified_attention for Attention.forward (vllm-project#11967)
0f8cafe 
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
@ywang96 @DarkLight1337
[Doc][V1] Update model implementation guide for V1 support (vllm-proj…
cd82499 
…ect#11998)

Signed-off-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
@hmellor
[Doc] Organise installation documentation into categories and tabs (v…
e8c23ff 
…llm-project#11935)

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@youkaichao
[platform] add device_control env var (vllm-project#12009)
458e63a 
Signed-off-by: youkaichao <youkaichao@gmail.com>
@shen-shanshan
[Platform] Move get_punica_wrapper() function to Platform (vllm-proje…
a7d5968 
…ct#11516)

Signed-off-by: Shanshan Shen <467638484@qq.com>
@e1ijah1
bugfix: Fix signature mismatch in benchmark's get_tokenizer function (
c6db213 
vllm-project#11982)

Signed-off-by: elijah <f1renze.142857@gmail.com>
@gshtras
Merge remote-tracking branch 'upstream/main'
ce53f46
@gshtras
Using list
5a51290
@gshtras
Revert "[misc] improve memory profiling (vllm-project#11809)"
079750e 
This reverts commit 889e662.
@gshtras
Trying to make scales work with compileable attention
043c93d
@gshtras
Docs lint
16f8680
@gshtras
Merge remote-tracking branch 'origin/main' into upstream_merge_25_01_13
eb4abfd
@gshtras gshtras merged commit 5976f48 into main Jan 14, 2025
21 of 26 checks passed
@gshtras gshtras deleted the upstream_merge_25_01_13 branch January 14, 2025 20:51
# for free to join this conversation on GitHub. Already have an account? # to comment
Reviewers

@shajrawi shajrawi shajrawi approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.