Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

[Bug]: Extra fields still not working #12864

Open
LianxinGao opened this issue Feb 7, 2025 · 2 comments
Open

[Bug]: Extra fields still not working #12864

LianxinGao opened this issue Feb 7, 2025 · 2 comments
Labels
bug Something isn't working

Comments

@LianxinGao
Copy link

LianxinGao commented Feb 7, 2025

I have already reviewed the PR at #10463, but this issue still persists.

docker:vllm/vllm-openai:latest

parameters:--model /data/models/mistralai/Mistral-Large-Instruct-2411 --disable-log-requests --trust-remote-code --enforce-eager --enable-auto-tool-choice --tool-call-parser mistral --tokenizer-mode mistral --config-format mistral

code:

from openai import OpenAI

client = OpenAI(base_url="http://xxx/v1",
                api_key="sk-xxx")

response = client.chat.completions.create(
    model="/data/models/mistralai/Mistral-Large-Instruct-2411",
    messages=[{"role": "user", "content": "hello", "name": "bob"}],
)

print(response.choices[0].message)

error: openai.BadRequestError: Error code: 400 - {'error': {'message': "1 validation error for ChatCompletionRequest\nmessages.0.user.name\n Extra inputs are not permitted [type=extra_forbidden, input_value='bob', input_type=str]\n For further information visit https://errors.pydantic.dev/2.10/v/extra_forbidden (request id: 2025020619014024915328971379754)", 'type': 'upstream_error', 'param': '400', 'code': 'bad_response_status_code'}}

@LianxinGao LianxinGao added the bug Something isn't working label Feb 7, 2025
@gcalmettes
Copy link
Contributor

gcalmettes commented Feb 7, 2025

see this comment

@patrickvonplaten any chance the current extra=forbid in the Mistral BaseModel could be updated to allow extra arguments ?

This would match the OpenAi policy on extra fields and ensure that the experience is the same for all models ran with vLLM.

@mahdi7
Copy link

mahdi7 commented Feb 19, 2025

The limitation of extra=forbid also blocks interaction of AutoGen agents with Mistral models served through vLLM.
@patrickvonplaten Are there any changes/updates planned for fixing this issue?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants