Skip to content

Commit 066e0d1

Browse files
robertgshaw2-redhatGWS0428
authored andcommitted
[V1] Bugfix: Validate Model Input Length (vllm-project#12600)
SUMMARY: * avoid crashing the engine when we get an input longer than max_model_len FIX vllm-project#12567(*link existing issues this PR will resolve*)
1 parent 73ec105 commit 066e0d1

File tree

1 file changed

+5
-0
lines changed

1 file changed

+5
-0
lines changed

vllm/v1/engine/processor.py

+5
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,11 @@ def _validate_model_inputs(self, inputs: ProcessorInputs):
206206
if prompt_ids is None or len(prompt_ids) == 0:
207207
raise ValueError("Prompt cannot be empty")
208208

209+
if len(prompt_ids) >= self.model_config.max_model_len:
210+
raise ValueError(
211+
f"Prompt length of {len(prompt_ids)} is longer than the "
212+
f"maximum model length of {self.model_config.max_model_len}.")
213+
209214
if self.model_config.is_multimodal_model:
210215
max_prompt_len = self.model_config.max_model_len
211216

0 commit comments

Comments
 (0)