Additional properties from OpenAI-compatible response are needed but not exposed

[stephentoub]

Service

OpenAI

Describe the bug

openai-dotnet/src/OpenAI.csproj

Lines 54 to 57 in ef201cd

	<InternalsVisibleTo Include="Azure.AI.OpenAI" Condition="'$(Configuration)' != 'Unsigned'">
	<PublicKey>0024000004800000940000000602000000240000525341310004000001000100097ad52abbeaa2e1a1982747cc0106534f65cfea6707eaed696a3a63daea80de2512746801a7e47f88e7781e71af960d89ba2e25561f70b0e2dbc93319e0af1961a719ccf5a4d28709b2b57a5d29b7c09dc8d269a490ebe2651c4b6e6738c27c5fb2c02469fe9757f0a3479ac310d6588a50a28d7dd431b907fd325e18b9e8ed</PublicKey>
	</InternalsVisibleTo>
	<InternalsVisibleTo Include="Azure.AI.OpenAI" Condition="'$(Configuration)' == 'Unsigned'" />

The OpenAI library is tracking all received properties and storing them into a SerializedAdditionalRawData dictionary. It doesn't publicly expose this data, except via InternalsVisibleTo to Azure.AI.OpenAI.

Why is Azure.AI.OpenAI special here, such that this data is only available to that specific library?

Steps to reproduce

Try to access additional properties sent down as part of an OpenAI-compatible response. You can't.

Code snippets

No response

OS

Any

.NET version

Any

Library version

2.1.0-beta

[KrzysztofCwalina]

We will be exposing this publicly. We just did not have time to design it properly in v1.

[JadynWong]

Expect extra_body, extra_headers to be supported like the Python version.

[fartwhif]

can't wait

I understand everything is beta and preview and "happening". But I'm just saying that the "you've been rate limited" failure information is extra important for those using realtime with only 100 RPD Thanks for your time.

ConversationResponseFinishedUpdate.StatusDetails.SerializedAdditionalRawData

{{"type":"requests","code":"rate_limit_exceeded","message":"Rate limit reached for 
gpt-4o-realtime-preview in organization org-XXXXXXXXXXXXXXXXXXXXXXXXXX on requests 
per day (RPD): Limit 100, Used 100, Requested 1. Please try again in 14m24s. 
Visit https://platform.openai.com/account/rate-limits to learn more."}}

[jichulu]

When reading the response from DeepSeek-R1, I need to get the reasoning_content property from SerializedAdditionalRawData.
Currently, I'm reading the value of SerializedAdditionalRawData via reflection, which may have a performance issue.
SerializedAdditionalRawData needs to be exposed as public.

var chatUpdate = client.CompleteChatStreamingAsync(Messages, options, cancellationToken);
await foreach (var item in chatUpdate)
{
    try
    {
		var additionalRawData = item.GetAdditionalRawData();
		if (additionalRawData?.TryGetValue("reasoning_content", out var val) == true)
		{
			onStreaming?.Invoke(val.ToString(), "thinking");
		}
    }
    catch (Exception)
    {
    }
}

public static IDictionary<string, BinaryData>? GetAdditionalRawData(this StreamingChatCompletionUpdate item)
{
	var choice = item.GetPrivatePropertyValue<object>("InternalChoiceDelta");
	return choice?.GetPrivatePropertyValue<IDictionary<string, BinaryData>>("SerializedAdditionalRawData");
}

Sample SSE

data:{"id":"4ae38986-9290-41b2-90cb-29b2b07d57a7","object":"chat.completion.chunk","created":1737779348,"model":"deepseek-reasoner","system_fingerprint":"fp_1c5d8833bc","choices":[{"index":0,"delta":{"role":"assistant","content":null,"reasoning_content":""},"logprobs":null,"finish_reason":null}],"usage":null}
data:{"id":"4ae38986-9290-41b2-90cb-29b2b07d57a7","object":"chat.completion.chunk","created":1737779348,"model":"deepseek-reasoner","system_fingerprint":"fp_1c5d8833bc","choices":[{"index":0,"delta":{"content":null,"reasoning_content":"OK"},"logprobs":null,"finish_reason":null}],"usage":null}
data:{"id":"4ae38986-9290-41b2-90cb-29b2b07d57a7","object":"chat.completion.chunk","created":1737779348,"model":"deepseek-reasoner","system_fingerprint":"fp_1c5d8833bc","choices":[{"index":0,"delta":{"content":null,"reasoning_content":"，"},"logprobs":null,"finish_reason":null}],"usage":null}

[edobnet]

I need too

[ZhangQueque]

Urgent repair, urgent, urgent

[dnalneh]

Nuget package v 2.2.0-beta.4

Related issue: I need annotation data from a responses api streaming request (i am using some additional uploaded files, that openai should look into and I need the filename). I have to use reflection to access SerializedAdditionalRawData in order to get the private dictionary, where the filename can be found:

await foreach (var chatUpdate in _openAIResponsesClient.CreateResponseStreamingAsync(message, new ResponseCreationOptions
 {
     PreviousResponseId = previousResponseId,   
     Tools = { _fileSearcher },
 }))
 {
     if (chatUpdate is StreamingResponseOutputTextDeltaUpdate outputTextUpdate)
     {                    
         await httpResponse.WriteAsync(outputTextUpdate.Delta);
     }
     else if (chatUpdate is StreamingResponseTextAnnotationAddedUpdate annotationAddedUpdate)
     {
         // Get the private property "SerializedAdditionalRawData"
         var annotationType = annotationAddedUpdate.Annotation.GetType();
         
         var propertyInfo = annotationType.GetProperty("SerializedAdditionalRawData", BindingFlags.NonPublic | BindingFlags.Instance);

         if (propertyInfo != null)
         {
             // Get the value of the private property
             var serializedAdditionalRawData = propertyInfo.GetValue(annotationAddedUpdate.Annotation) as IDictionary<string, BinaryData>;
             
             if (serializedAdditionalRawData != null)
             {
                 foreach (var kvp in serializedAdditionalRawData)
                 {
                     myValues.Add(kvp.Value.ToString());
                 }
             }
             else
             {
                 Console.WriteLine("SerializedAdditionalRawData is null or not a dictionary.");
             }
         }
         else
         {
             Console.WriteLine("Property 'SerializedAdditionalRawData' not found.");
         }
     }  
 }

Would be nice to have normal access to this data.