Skip to content

Missing Handling for delta.reasoning_content in agents.models.chatcmpl_stream_handler.ChatCmplStreamHandler.handle_stream #578

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Open
kingmming opened this issue Apr 23, 2025 · 5 comments
Labels
bug Something isn't working

Comments

@kingmming
Copy link

Issue Title: Missing Handling for delta.reasoning_content in agents.models.chatcmpl_stream_handler.ChatCmplStreamHandler.handle_stream

Description:

In the ChatCmplStreamHandler.handle_stream method, the case where delta.reasoning_content is present is not handled. This could lead to incomplete or incorrect handling of the stream response when reasoning content is provided by the model.

Problematic Code Example:

class ChatCmplStreamHandler:
    @classmethod
    async def handle_stream(
        cls,
        response: Response,
        stream: AsyncStream[ChatCompletionChunk],
    ) -> AsyncIterator[TResponseStreamEvent]:
        usage: CompletionUsage | None = None
        state = StreamingState()

        async for chunk in stream:
            
            if not state.started:
                state.started = True
                yield ResponseCreatedEvent(
                    response=response,
                    type="response.created",
                )

            usage = chunk.usage

            if not chunk.choices or not chunk.choices[0].delta:
                continue

            delta = chunk.choices[0].delta

            # Handle text
            if delta.content:
		…… 

            # Handle refusals (model declines to answer)
            if delta.refusal:

Steps to Reproduce:

  1. Use a model that provides reasoning content in its responses.
  2. Trigger a stream response that includes delta.reasoning_content.
  3. Observe that the ChatCmplStreamHandler.handle_stream method does not process or emit events related to the reasoning content.

Actual Behavior:

The delta.reasoning_content is ignored, and no events are emitted to handle or notify consumers of the reasoning content.

Impact:

This issue prevents the proper handling and display of reasoning content in the stream, which is crucial for models that provide explanations or reasoning behind their responses.

Proposed Solution:

Add handling for delta.reasoning_content in the ChatCmplStreamHandler.handle_stream method. Example code snippet:

class ChatCmplStreamHandler:
    @classmethod
    async def handle_stream(
        cls,
        response: Response,
        stream: AsyncStream[ChatCompletionChunk],
    ) -> AsyncIterator[TResponseStreamEvent]:
        usage: CompletionUsage | None = None
        state = StreamingState()

        async for chunk in stream:
            
            if not state.started:
                state.started = True
                yield ResponseCreatedEvent(
                    response=response,
                    type="response.created",
                )

            usage = chunk.usage

            if not chunk.choices or not chunk.choices[0].delta:
                continue

            delta = chunk.choices[0].delta

            # Handle text
            if delta.content:

            # Handle reasoning content text
            if delta.reasoning_content:
   
@kingmming kingmming added the bug Something isn't working label Apr 23, 2025
@rm-openai
Copy link
Collaborator

@kingmming which model produces delta.reasoning_content?

@kingmming
Copy link
Author

@kingmming which model produces delta.reasoning_content?

Hi @rm-openai ! The delta.reasoning_content is generated by the DeepSeek-R1 model.

When the enable-reasoning option is enabled during the OpenAI-compatible service startup, the DeepSeek-R1 model provides reasoning content in the response stream.

Here is an example of how it appears in the response data:

{
    "choices": [
        {
            "delta": {
                "content": null,
                "reasoning_content": " smoothly"
            },
            "finish_reason": null,
            "index": 0,
            "logprobs": null
        }
    ],
    "object": "chat.completion.chunk",
    "usage": null,
    "created": 1745454612,
    "system_fingerprint": null,
    "model": "DeepSeek-R1",
    "id": "chatcmpl-281b5461-bfa3-99f5-b3cb-650b2306e712"
}
{
    "choices": [
        {
            "delta": {
                "content": " assistant",
                "reasoning_content": null
            },
            "finish_reason": null,
            "index": 0,
            "logprobs": null
        }
    ],
    "object": "chat.completion.chunk",
    "usage": null,
    "created": 1745454612,
    "system_fingerprint": null,
    "model": "DeepSeek-R1",
    "id": "chatcmpl-281b5461-bfa3-99f5-b3cb-650b2306e712"
}

@Ddper
Copy link
Contributor

Ddper commented Apr 24, 2025

I'm working on this issue, see this PR #494

@Burlesque1
Copy link

Burlesque1 commented May 9, 2025

`
@classmethod
def message_to_output_items(cls, message: ChatCompletionMessage) -> list[TResponseOutputItem]:
items: list[TResponseOutputItem] = []
message_item = ResponseOutputMessage(
id=FAKE_RESPONSES_ID,
content=[],
role="assistant",
type="message",
status="completed",
)
if message.content:
message_item.content.append(
ResponseOutputText(text=message.content, type="output_text", annotations=[])
)
if message.refusal:
message_item.content.append(
ResponseOutputRefusal(refusal=message.refusal, type="refusal")
)
if message.audio:
raise AgentsException("Audio is not currently supported")

    if message_item.content:
        items.append(message_item)

    if message.tool_calls:
        for tool_call in message.tool_calls:
            items.append(
                ResponseFunctionToolCall(
                    id=FAKE_RESPONSES_ID,
                    call_id=tool_call.id,
                    arguments=tool_call.function.arguments,
                    name=tool_call.function.name,
                    type="function_call",
                )
            )
    
    # if message.reasoning_content:
        
    #     items.append(

    return items`

here is also the missing part of reasoning_content

@dongxiaohuang
Copy link

any updates?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants