Caching disposable objects in memory

[esskar]

I am caching HttpResponseMessages using custom serialzer/deserialzer

public sealed class HttpMessageSerializer(ILogger<HttpMessageSerializer> logger) : IFusionCacheSerializer
{
    private readonly FusionCacheProtoBufNetSerializer _serializer = new();

    public async ValueTask<byte[]> SerializeAsync<T>(T? obj, CancellationToken token = default)
    {
        if (obj is not FusionCacheDistributedEntry<HttpResponseMessage> entry)
            throw new NotSupportedException("Only HttpResponseMessage can be serialized.");
        
        // ensure we have the content buffered, so we know the content length
        await entry.Value.Content.LoadIntoBufferAsync(token);
        
        RemoveChunkedTransferEncoding(entry.Value, out var contentLength);
        RemoveNonDeserializableHeaders(entry.Value);
        
        using var httpMessageContent = new HttpMessageContent(entry.Value);
        await using var contentStream = await httpMessageContent.ReadAsStreamAsync(token);
        await using var stream = new ArrayPoolWritableStream();
        await contentStream.CopyToAsync(stream, token);
        
        var contentType = entry.Value.Content.Headers.ContentType?.MediaType ?? MediaTypeApplicationOctetStream;
        logger.LogSerializedResponse(contentLength, contentType, contentStream.Length);

        var clonedEntry = new FusionCacheDistributedEntry<byte[]>
        {
            LogicalExpirationTimestamp = entry.LogicalExpirationTimestamp,
            Metadata = entry.Metadata,
            Tags = entry.Tags,
            Timestamp = entry.Timestamp,
            Value = stream.GetBytes()
        };
        
        return await _serializer.SerializeAsync(clonedEntry, token);
    }

    public async ValueTask<T?> DeserializeAsync<T>(byte[] data, CancellationToken token = default)
    {
        if (typeof(T) != typeof(FusionCacheDistributedEntry<HttpResponseMessage>))
            throw new NotSupportedException("Only HttpResponseMessage can be deserialized.");
        
        var entry = await _serializer.DeserializeAsync<FusionCacheDistributedEntry<byte[]>>(data, token);
        if (entry is null)
            return default;

        using var response = new HttpResponseMessage();
        response.Content = new ByteArrayContent(entry.Value);
        response.Content.Headers.Add(HeaderNames.ContentType, MediaTypeApplicationHttpResponse);
        
        var responseMessage = await response.Content.ReadAsHttpResponseMessageAsync(token);
        await responseMessage.Content.LoadIntoBufferAsync(token);
        
        var result = new FusionCacheDistributedEntry<HttpResponseMessage>
        {
            LogicalExpirationTimestamp = entry.LogicalExpirationTimestamp,
            Metadata = entry.Metadata,
            Tags = entry.Tags,
            Timestamp = entry.Timestamp,
            Value = responseMessage
        };
        return (T)(object)result;
    }
    
    private static void RemoveChunkedTransferEncoding(HttpResponseMessage response, out long? contentLength)
    {
        contentLength = response.Content.Headers.ContentLength;
        if (response.Headers.TransferEncodingChunked != true || !contentLength.HasValue) 
            return;
        
        response.Headers.TransferEncodingChunked = false;
        response.Content.Headers.ContentLength = contentLength;
    }
    
    private static void RemoveNonDeserializableHeaders(HttpResponseMessage response)
    {
        response.Headers.Remove(HeaderNames.ETag);
    }
    
    private const string MediaTypeApplicationOctetStream = "application/octet-stream";
    private const string MediaTypeApplicationHttpResponse = "application/http; msgtype=response";
    
    public byte[] Serialize<T>(T? obj) => throw new NotSupportedException("Use async method instead.");

    public T? Deserialize<T>(byte[] data) => throw new NotSupportedException("Use async method instead.");
}

this is used inside a DelegationHandler

    protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
    {
        if (request.Method != HttpMethod.Get)
            return await base.SendAsync(request, cancellationToken);

        var cacheKey = _cacheKeyFactory.Create(request.RequestUri!.ToString().ToLowerInvariant());
        
        var response = await _fusionCache.GetOrSetAsync<HttpResponseMessage>(cacheKey, async (context, token) =>
        {
            var response = await base.SendAsync(request, token);
            if (response.IsSuccessStatusCode) 
                return response;
            
            // Skip caching if the response is not successful
            context.Options.SetSkipMemoryCache();
            context.Options.SetSkipDistributedCache(true, true);
            return response;
        }, token: cancellationToken);

        return response;
    }

while this works distributed cache calls, the memory cache does not use the serialzer, but keeps trying to reuse the HttpResponseMessage from memory; this forces me to SkipMemoryCacheRead = true and SkipMemoryCacheWrite = true

Are there better ways?

[jodydonetti]

Hi @esskar try to take a look at Auto-Clone.

Hope this helps, let me know.

[esskar]

Thanks. I kinda get it to work. Is there a way to need to implement the sync functions of IFusionCacheSerializer?

[jodydonetti]

Sorry, can you clarify?

[esskar]

When turning on AutoClone, the library calls the Serialize/Deserialize functions, even tough i am only using the GetOrSetAsync on the IFusionCache interface. The serializaiom/deserialization of HttpResponseMessage currently only works performant when doing async operations.

[jodydonetti]

Ah, I see what you meant now.

When turning on AutoClone, the library calls the Serialize/Deserialize functions even tough i am only using the GetOrSetAsync

I don't know if this makes a difference in your case, but one thing to keep in mind is that Serialize() is only called once, and is Deserialize() which is called every time.

The serializaiom/deserialization of HttpResponseMessage currently only works performant when doing async operations.

Since deserialization in this case starts from an already allocated byte[] can I suggest to benchmark if it is in fact more peformant in the async way? I'm asking because it is a known fact that it's very frequently more performance to do sync (de)serialization, as it does need to allocate and handle the async state machinery.
For example the new IHybridCacheSerializer only has the sync versions.

Can you do a benchmark on both sync and async versions? That would be great.

Meanwhile I'll take a look at what it would mean to use the async version in those code paths.

Thanks.