Skip to content

Streaming Fails Due to {"include_usage": True} #442

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom
Open
sixwaaaay opened this issue Apr 5, 2025 · 2 comments
Open

Streaming Fails Due to {"include_usage": True} #442

sixwaaaay opened this issue Apr 5, 2025 · 2 comments

Comments

@sixwaaaay
Copy link

Description
When using Mistral AI's OpenAI-compatible API with streaming output Runner.run_streamed, the hardcoded stream_options={"include_usage": True} in the code triggers a 422 Unprocessable Entity error. Mistral's API does not support the include_usage parameter in stream_options, causing compatibility issues.

Root Cause:

openai-agents-python/src/agents/models/openai_chatcompletions.py

Line 539 in 064e25b

stream_options={"include_usage": True} if stream else NOT_GIVEN,

Proposed Fix
Add an override mechanism for stream_options

Error

openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'stream_options', 'include_usage'], 'msg': 'Extra inputs are not permitted', 'input': True}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}
@rm-openai
Copy link
Collaborator

Proposed fix makes sense, PR welcome!

@Ddper Ddper mentioned this issue Apr 10, 2025
Merged
@Ddper
Copy link
Contributor

Ddper commented Apr 10, 2025

Proposed fix makes sense, PR welcome!

@rm-openai I created a PR, pls review.

rm-openai pushed a commit that referenced this issue Apr 10, 2025
@Ddper
add overwrite mechanism for stream_options (#465)
8ded8a9 
fix issue #442

below is an example to overwrite include_usage


```
    result = Runner.run_streamed(
        agent,
        "Write a haiku about recursion in programming.",
        run_config=RunConfig(
            model_provider=CUSTOM_MODEL_PROVIDER,
            model_settings=ModelSettings(include_usage=True)
        ),
    )
```
Lightblues pushed a commit to Lightblues/openai-agents-python that referenced this issue Apr 13, 2025
@Ddper
add overwrite mechanism for stream_options (openai#465)
b1b7b44 
fix issue openai#442

below is an example to overwrite include_usage


```
    result = Runner.run_streamed(
        agent,
        "Write a haiku about recursion in programming.",
        run_config=RunConfig(
            model_provider=CUSTOM_MODEL_PROVIDER,
            model_settings=ModelSettings(include_usage=True)
        ),
    )
```
@cnm13ryan cnm13ryan mentioned this issue Apr 25, 2025
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Ddper @sixwaaaay @rm-openai