ClientWebSocket - MemoryStream, message framing and performance

Question

This is more of a theoretical question around possible performance optimizations when using C# ClientWebSocket. Consider following implementation for receiving socket messages.

private async Task ReceiveLoop(CancellationToken cancellationToken, int receiveBufferSize)
{
    try
    {
        WebSocketReceiveResult receiveResult;
        ArraySegment<byte> buffer = new(new byte[receiveBufferSize]);
        do
        {
            await using var stream = new MemoryStream();

            do
            {
                receiveResult = await handler.ReceiveAsync(buffer, cancellationToken);
                await stream.WriteAsync(buffer.AsMemory(0, receiveResult.Count), cancellationToken);

            } while (!receiveResult.EndOfMessage);

            if (receiveResult.MessageType == WebSocketMessageType.Close)
            {
                break;
            }
            var arrayCopy = stream.ToArray();
            // Add to MessageQueue(arrayCopy)
            // Somewhere in the code Consumer, reads the queue and deserializes the array to POCO a few microseconds later
        }
        while (!cancellationToken.IsCancellationRequested);
    }
    catch (OperationCanceledException)
    {
        await DisconnectAsync(cancellationToken);
    }
}

To put this to some constraints: The deserialization or say encoding to utf8string (essentially any copy) from stream here seems to be what most implementations look like. However, if the stream is receiving high amount of events or for some reason deserialization failed it could potentially create congestion in the thread pool queue. So the current implementation adds to the queue and processes it by separate Task. As well has the logic around different message response types.

If there is no array copy, the stream is disposed and there for the deserialization will fail. Another approach is to throw more memory at the problem, say ArrayPool.Rent(81920) and only return the array after Consumer has deserialized. This reduces the load on CPU if we take out MemoryStream however puts pressure on the allocations and the GC. (As well doesn't expand for message frames if message > 81920).

• Is there a reasonable way to reuse the stream without a copy?
• Would you argue that "copy" to any kind at this point makes the most sense before the stream is disposed or array mutated?
• Are there better alternatives to using of MemoryStream? (RecycableMemoryStream.GetBuffer()?) that could potentially give an option to solve the problem?
• Is there any other option like holding on to the underlying MemoryStream buffer until the its processed in a safe manner?
• Would IO Pipelines help in this case and moving away from ClientWebSocket (having all its ping/pongs) is the best option here?
• What is the least improvement possible to this example to make it more efficient?

Answer 1

While there is a general rule to dispose everything disposable, this is much less important for MemoryStream. The only resource it owns is memory, and memory is managed by the garbage collector anyway, disposing it do not really do much. So the simplest option would be to just remove the await using and add the memory stream to the queue.

Another option could be Microsoft.IO.RecyclableMemoryStream. This do require disposing the of the streams, since it uses pooled buffers internally. And this should make it more GC friendly for short lived streams.

I'm not very familiar with websockets. But if you could get access to the message length when starting to receive the message you should be able to rent a suitably sized buffer from a memory pool. And this could potentially avoid most allocations and copies.

But before doing anything I would recommend doing some profiling and/or benchmarking to confirm you actually have a problem. While avoiding allocations is certainly a worthy goal, the GC is fairly forgiving.