Skip to content

Streaming Fails Due to {"include_usage": True} #442

Closed
@sixwaaaay

Description

@sixwaaaay

Description
When using Mistral AI's OpenAI-compatible API with streaming output Runner.run_streamed, the hardcoded stream_options={"include_usage": True} in the code triggers a 422 Unprocessable Entity error. Mistral's API does not support the include_usage parameter in stream_options, causing compatibility issues.

Root Cause:

stream_options={"include_usage": True} if stream else NOT_GIVEN,

Proposed Fix
Add an override mechanism for stream_options

Error

openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'stream_options', 'include_usage'], 'msg': 'Extra inputs are not permitted', 'input': True}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions