Unable to accurately determine SignalR message size and prevent InvalidDataException. Based on my testing, this is Blazor Server specific

[Bond-Addict]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Currently when getting and setting session variables I'm getting InvalidDataException($"The maximum message size of {maxMessageSize}B was exceeded. The message size can be configured in AddHubOptions."); thrown from the HubConnectionHandler. I do understand that the way to resolve this is to increase the message size. That being said, I only want to increase the necessary amount and not set some arbitrary size. I have tried inspecting the HubConnectionContext that is available at the time the exception is thrown and see something like

I've also dug into ProtectedBrowserStorage.cs to try and maybe just duplicate the encoding manually.

This is what I've done with my model to try and debug the values

Which I was assuming would work since

Is from SessionExtensions.cs.

Which doesnt seem to really add up either. When I set the MaximumReceiveMessageSize to 43000 which I would think would hold 42769 with no issue, I get the InvalidDataException.

I've also verified that when calling sessionStorage.getItem('myKey').length that the length is 42769. So I'm quite puzzled. Is there some way I can setup my application or some too I can acquire to measure my actual message size? I want to just up it to my current message size and then give it another kb for some breathing room.

My thought was that potentially _lastExaminedIndex + _uncomsumedBytes = 47898 which seemed right initially since that would be slightly larger than 42769 which gets kicked out from Encoding.UTF8.GetBytes(). I've been incrementally increasing and decreasing the MaximumReceiveMessageSize to try and hone in on the actual exact size. If the calculation was really precise, I would expect 47898 and 47899 to not throw the exception, but 47897 would. I would think it would be pretty precise since DispatchMessagesAsync is using if (segment.Length > maxMessageSize) to determine if it should throw the InvalidDataException.

If my model size before my model bytes in razor code behind is 42769 and store item size in the browser is 42796 why on earth wouldn't a MaximumReceiveMessageSize of 43kb be able to hold it?

Also, if I'm able to successfully save my item to session of 42796 it doesn't make sense that it will throw InvalidDataException when retrieving the item from session storage.

I've tried to be a thorough in my investigations and although the code is very clean and easy to read, the level of abstraction and bouncing between the different Microsoft repo's (like trying to understand what PipeReader used on the DispatchMessagesAsync in HubConnectionHandler which is held in System.IO.Pipelines) it's very likely I've missed something easy and obvious.

Thanks!

P.S. I know that the easy solution is to just set MaximumReceiveMessageSize to a high number, but to me that feels like just throwing parts at a broken car which may fix it, but then you'll never know what broke in the first place making it harder to fix the same issue later. I want to make reasonable, calculated adjustments.

Expected Behavior

If my model byte size before sending to SignalR is 42769 and the session item successfully stored is 42769 then if I have a MaximumReceiveMessageSize of 43000 then both setting and getting session items would succeed.

Steps To Reproduce

No response

Exceptions (if any)

InvalidDataException($"The maximum message size of {maxMessageSize}B was exceeded. The message size can be configured in AddHubOptions.");

.NET Version

9.0.100 but this also happen in 8x

Anything else?

I've tested in brand new 9x and 8x Blazor server and web assembly projects. So far, it seems that only Blazor Server experiences the issue.

[Bond-Addict]

Also, I feel that only increasing the message size for what is actually needed follows SignalR/SignalR#1205 (comment)

[mkArtakMSFT]

Thanks for contacting us.
As you've already realized, this is by-design. The payload content is not the only content to be sent and hence, more data will need to be sent, which in-turn, requires more capacity. Unfortunately, we don't have a simple way to calculate the precise number here. So, targeting to have something like +10-20% capacity is the safe thing to do.

[Bond-Addict]

Thanks for contacting us. As you've already realized, this is by-design. The payload content is not the only content to be sent and hence, more data will need to be sent, which in-turn, requires more capacity. Unfortunately, we don't have a simple way to calculate the precise number here. So, targeting to have something like +10-20% capacity is the safe thing to do.

Can you clarify "So, targeting to have something like +10-20% capacity is the safe thing to do."

Are you saying that I should add 10-20% of my Encoding.UTF8.GetBytes() which currently comes out to 42769 bytes? Making the new capacity 47,045.9 - 51,322.8 bytes? Or is a different base metric I should add to?

[Bond-Addict]

Thanks for contacting us. As you've already realized, this is by-design. The payload content is not the only content to be sent and hence, more data will need to be sent, which in-turn, requires more capacity. Unfortunately, we don't have a simple way to calculate the precise number here. So, targeting to have something like +10-20% capacity is the safe thing to do.

Is by design that even when enabling trace level logging that the actual message size sent isnt logged out as well? Thus far I haven't seen anyway to get that info logged out via the debug console, event viewer or any other mechanic.

[Bond-Addict]

Thanks for contacting us. As you've already realized, this is by-design. The payload content is not the only content to be sent and hence, more data will need to be sent, which in-turn, requires more capacity. Unfortunately, we don't have a simple way to calculate the precise number here. So, targeting to have something like +10-20% capacity is the safe thing to do.

Additionally, with that being the case, could that info in some fashion be added to the blazor and or signal r docs? I spent quite sometime and couldn't find that answer. If I perhaps already available and I missed it, can you maybe point me to its location?

I've already tried following things like
https://www.appvnext.com/blog/2019/12/21/logging-signalr-traffic-in-server-side-blazor

https://learn.microsoft.com/en-us/aspnet/signalr/overview/testing-and-debugging/enabling-signalr-tracing

https://learn.microsoft.com/en-us/aspnet/core/signalr/diagnostics?view=aspnetcore-9.0

All of which I've been able to get to work in both 8x and 9x Blazor Server

[Bond-Addict]

@mkArtakMSFT can you maybe help explain why I can have a object that is 42k bytes (which is more than the 32k default limit), I can save the object and it shows in session, I can then try and save again and only then it errors. The object is the same size on both saves. Also, I can save the object the first time, then trigger getting the item from session and then it will error.

It's doesn't make sense to me that

1. If the object is too big the first time, that it wouldnt initially error and not save on the first attempt
2. Save just fine and then crash trying to get the item it already successfully saved.