JsonElement GetRawText method throws "Operation is not valid" exception on empty array

Question

Situation

I am running into an issue with my implementation of .NET Core's System.Text.Json.JsonSerializer. The API which my application utilizes for getting data returns JSON in the following format:

{
    "context": "SomeUnusedContextValue",
    "value": [
        {...}
    ]
}

I only care for the actual response, so I only need the value item.

I have written below method to get the specific property and deserialize the items into objects.

 public static async Task<T?> DeserializeResponse<T>(Stream str, string? property, CancellationToken ct)
 {
        JsonDocument jsonDocument = await JsonDocument.ParseAsync(str, default, ct).ConfigureAwait(false);

        if (property is null) // some calls to the API do return data at root-level
        {
            return JsonSerializer.Deserialize<T>(jsonDocument.RootElement.GetRawText());
        }

        if (!jsonDocument.RootElement.TryGetProperty(property, out JsonElement parsed))
            throw new InvalidDataException($"The specified lookup property \"{property}\" could not be found.");

        return JsonSerializer.Deserialize<T>(!typeof(IEnumerable).IsAssignableFrom(typeof(T))
            ? parsed.EnumerateArray().FirstOrDefault().GetRawText()
            : parsed.GetRawText(), new JsonSerializerOptions
            { DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault });
}

Problem

Now for my problem. Sometimes I only need a single object, however even if there is only one result, the API will still return an array. Not a problem since, as can be seen in the bottom return statement, I will just enumerate the array and get the first item (or the default null). This seems to crash when the returned array is empty, throwing below exception:

System.InvalidOperationException: Operation is not valid due to the current state of the object.
   at System.Text.Json.JsonElement.GetRawText()
   at BAS.Utilities.Deserializing.ResponseDeserializer.DeserializeResponse[T](Stream str, String property, CancellationToken ct) in C:\dev\bas.api\Modules\BAS.Utilities\Deserializing\ResponseDeserializer.cs:line 40

The object I'm trying to serialize into is as below:

public class JobFunctionCombination
{
        /// <summary>
        /// Gets or sets the combined identifier of the main function group, the function group and the sub function group.
        /// </summary>
        /// <example>01_0101_010101</example>
        [JsonPropertyName("job_function_combination_id")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public string Id { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the combined names of the function groups.
        /// </summary>
        /// <example>Management | Human Resources | Finance controller</example>
        [JsonPropertyName("description")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public string Description { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the identifier of the main function group.
        /// </summary>
        [JsonPropertyName("job_main_function_group_id")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public string MainFunctionGroupId { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the identifier of the function group.
        /// </summary>
        [JsonPropertyName("job_function_group_id")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public string FunctionGroupId { get; set; } = string.Empty;

        /// <summary>
        /// Gets or sets the identifier of the sub function group.
        /// </summary>
        [JsonPropertyName("job_sub_function_group_id")]
        [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingDefault)]
        public string SubFunctionGroupId { get; set; } = string.Empty;
}

The types and JsonPropertyName attributes all match to the returned JSON.

Attempted fixes

To try and fix this issue, I have tried some fixes (two of which you can still see in the given code samples).

• Adding the JsonIgnore attribute to the properties in the class.
  • I have tried to set the condition to WhenWritingDefault as well as WhenWritingNull. Neither seem to fix the problem.
• Setting the DefaultIgnoreCondition in a JsonSerializerOptions object passed to JsonSerializer.Deserialze.
  • Here I have also tried both WhenWritingDefault and WhenWritingNull, also to no avail.
• Checking if the array in JsonElement, when enumerated, is null or empty using .isNullOrEmpty().
  • This does prevent the exception from occuring, however it does not seem like an actual fix to me. More like a FlexTape solution just cancelling out the exception.

I am not sure what the exact issue is, other than the clear fact that JsonSerializer clearly has an issue with null objects. What could I do to fix this?

Answer 1

Your problem is with your call to FirstOrDefault():

parsed.EnumerateArray().FirstOrDefault()

JsonElement is a struct, so when the array has no items, FirstOrDefault() will return a default struct -- one initialized with zeros but without any property values set. Such an element does not correspond to any JSON token; ValueKind will have the default value of JsonValueKind.None and GetRawText() will have no raw text to return. In such a situation Microsoft chose to make GetRawText() throw an exception rather than return a null string.

To avoid the problem, enumerate the array, use a Select() statement to deserialize each item, then afterwards do FirstOrDefault() to return default(T) when the array is empty, like so:

public static partial class JsonExtensions
{
    public static async Task<T?> DeserializeResponse<T>(Stream str, string? property, CancellationToken ct = default)
    {
        if (property is null) // some calls to the API do return data at root-level
        {
            return await JsonSerializer.DeserializeAsync<T>(str, cancellationToken: ct).ConfigureAwait(false);
        }

        using var jsonDocument = await JsonDocument.ParseAsync(str, default, ct).ConfigureAwait(false);

        if (!jsonDocument.RootElement.TryGetProperty(property, out JsonElement parsed))
            throw new InvalidDataException($"The specified lookup property \"{property}\" could not be found.");

        return typeof(T).IsSerializedAsArray() 
            ? parsed.Deserialize<T>()
            : parsed.EnumerateArray().Select(i => i.Deserialize<T>()).FirstOrDefault();
    }       
    
    static bool IsSerializedAsArray(this Type type) =>
        type != typeof(string)
        && typeof(IEnumerable).IsAssignableFrom(type)
        && type != typeof(byte []) // byte arrays are serialized as Base64 strings.
        && !type.IsDictionary();

    static bool IsDictionary(this Type type) =>
        typeof(IDictionary).IsAssignableFrom(type)
        || type.GetInterfacesAndSelf().Any(t => t.IsGenericType && t.GetGenericTypeDefinition() == typeof(IDictionary<,>));

    static IEnumerable<Type> GetInterfacesAndSelf(this Type type) =>
        (type ?? throw new ArgumentNullException()).IsInterface 
        ? new[] { type }.Concat(type.GetInterfaces())
        : type.GetInterfaces();
}

Notes:

• Since Enumerable.Select() uses deferred execution, only the first element (if present) will be deserialized.

• In .NET 6 you may use JsonSerialzier.Deserialize(JsonElement, Type, JsonSerializerOptions = default) to deserialize directly from a JsonElement. In .NET 5 and earlier, use one of the ToObject<T>() workaround extension methods from System.Text.Json.JsonElement ToObject workaround.

• I corrected the logic that tests whether the type T is to be serialized as an array.

• JsonDocument is disposable, and must be disposed of to return pooled memory to the memory pool.

Demo fiddle here.