Announcing dynamic JSON in the Azure Core library for .NET

Anne Thompson

More and more, Azure SDK client libraries are adding protocol methods—service methods that return a raw Response instead of the typical strongly typed model contained in Response<T>. When calling these methods, you can parse the response content using any JSON parser, such as JsonDocument. Use of existing APIs, however, can result in code that’s difficult to read and may not cleanly express your intent as an author. To solve this problem, the latest release of Azure.Core introduces a way to get response content as a dynamic type. This feature lets you write code at the protocol layer that has the same look and feel as the code you write using a client’s convenience layer.

The ToDynamicFromJson method

The Content property on Response is a BinaryData. We’ve added an extension method to BinaryData called ToDynamicFromJson. This method returns an instance of a new type called DynamicData that lets you treat JSON content like a strongly typed model. For example, if your response holds a configuration setting represented by the JSON { "value": "Cyan" }, the following code retrieves its value property as a string:

dynamic setting = response.Content.ToDynamicFromJson();
string value = setting.value;
Console.WriteLine($"Setting value: {setting.value}");

// Prints:
// Setting value: Cyan

The dynamic JSON that is returned is also mutable. You can modify its properties or even add new properties to it. For example, if you wanted to change the color of your configuration setting, you could modify it directly:

dynamic setting = response.Content.ToDynamicFromJson();
setting.value = "Green";
Console.WriteLine($"Setting value: {setting.value}");

// Prints:
// Setting value: Green

You might notice these samples don’t look exactly like what you’d write if you used a model type instead of a raw response. That’s because .NET types don’t use camel-case property names. ToDynamicFromJson lets you fix that. You can pass a propertyNameFormat parameter to the method, telling it what naming convention your JSON property names use. If you do, DynamicData will convert your C# property names to names that match the JSON content. With this option, we can rewrite the previous samples to look more like idiomatic C#. Since the sample JSON uses camel-cased names, we’ll pass JsonPropertyNames.CamelCase into the method:

dynamic setting = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);
string value = setting.Value;
setting.Value = "Green";

Now, your code can use standard C#-style conventions and still read and write JSON that uses any convention supported in the JsonPropertyNames enum. Admittedly, today only camel-case is supported, but it’s entirely possible we would support snake-case or another property name format used by Azure services in the future.

Round-tripping values

If you write many apps using Azure services, you may have come across a case similar to the last example. You need to retrieve a value from a service, change one or two fields, and send it back to the service to store your updates. On the Azure SDK team, we call this case a “round-trip scenario.”

The convenience methods on our client libraries make it easy to get a model, set some properties, and then call an update method, passing in the model you changed. For example, using our .NET library for the Azure App Configuration service, we might update an application setting that controls our app’s font color:

ConfigurationSetting setting = client.GetConfigurationSetting("FontColor");
setting.Value = "Cyan";
client.SetConfigurationSetting(setting);

We feel that writing code for this scenario is relatively straightforward using convenience methods. If you needed to round-trip values using protocol methods alone, however, it could be a lot more complicated. The added complexity would be because, to create the update to send in the request, you would need to create an instance of an anonymous type:

JsonDocument doc = JsonDocument.Parse(response.Content);
string fontColor = doc.RootElement.GetProperty("value").GetString();

var updatedSetting = new
{
    value = "Cyan"
};

client.SetConfigurationSetting(
    "FontColor", RequestContent.Create(updatedSetting), ContentType.ApplicationJson);

With the anonymous type approach, to prevent accidentally overwriting fields stored on the service, you would need to ensure you included every property from the response content in the new request content. This makes the code you would write more complicated, and it would be easy to make a mistake that could result in data loss.

DynamicData solves this problem, and as previously mentioned, lets you write code that looks much like our first example using convenience methods:

dynamic setting = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);
setting.Value = "Cyan";
client.SetConfigurationSetting(
    "FontColor", RequestContent.Create(setting), ContentType.ApplicationJson);

You might wonder, “Couldn’t I do this using existing mutable JSON DOM APIs like JsonNode?” And it’s true, you can. The Response.Content value can be used any way you like. For example, this sample shows how to achieve the same functionality using JsonNode:

JsonNode setting = JsonNode.Parse(response.Content.ToStream());
setting["value"] = "Cyan";
client.SetConfigurationSetting(
    "FontColor", RequestContent.Create(setting), ContentType.ApplicationJson);

With this approach, though, if you later wanted to change your application to use a client’s model-based APIs, you’d need to rewrite these parts of your code. Our goal with DynamicData is to make it possible for you to use syntax similar to Azure SDK client convenience methods so that making this type of change is as simple as possible.

Behaving like an Azure SDK model type

We mentioned before that DynamicData lets you use C#-style conventions when accessing properties. This is part of a larger design goal we had for the type, which was for it to behave like an Azure SDK model type. Model-like behavior shows up in a few other ways as well, for example, in how DynamicData handles optional properties and how it handles date/time values.

In our SDK model types, there are sometimes properties that the service considers optional and may or may not send back in a given response. When a service doesn’t return a value for an optional property, the corresponding model properties are null. To check if an optional property has a value, you might do what’s shown in this Azure Text Analytics library sample:

if (entity.Assertion is not null)
{
    Console.WriteLine($"  Assertions:");
    if (entity.Assertion.Association is not null)
    {
        Console.WriteLine($"    Association: {entity.Assertion.Association}");
    }
}

To let you write the same code at the protocol layer, DynamicData returns null if you access a property that’s not in the JSON it’s holding. This is different from the behavior you would get from an API like JsonElement.GetProperty, which would throw if the property you asked for didn’t exist.

Similarly, we’ve customized how DateTime and DateTimeOffset values you set on an instance of dynamic content are serialized to match what our Azure model types do, which complies with the Azure service API guidelines. For the rare Azure service that uses Unix times, you can tell ToDynamicFromJson about that too, by passing a value of x for the dateTimeFormat parameter.

What are best practices for use?

Overall, we try to design our types so that you don’t have to understand much about their implementation details to use them, and that’s largely true in this case too. However, with DynamicData, knowing something about how the layers around your JSON content work might help you use it more efficiently.

We shared previously that ToDynamicFromJson returns an instance of DynamicData, which adds a dynamic layer around your JSON content. But how does it read into the JSON buffer, and how does it mutate the JSON? Did we put a dynamic wrapper around JsonNode?

The answer to the last question is no, and not because JsonNode isn’t a great type! It’s just that JsonNode has different memory usage characteristics than JsonDocument, and we wanted a type that approximates the memory usage patterns of JsonDocument. To achieve this goal, we created an internal type called MutableJsonDocument that adds a layer of mutability over JsonDocument. It does this using a change list implementation.

When you set a value on MutableJsonDocument, we add that value to a list of changes. Then, anytime we’re asked to read a value from the JSON content, we check to see if there are any changes to that value before retrieving it from the original buffer. Conceptually, our implementation looks a little like we’ve added the following methods to JsonElement:

void Set(string value)
{
    changes.AddChange(_pathToElement, value);
}

string GetString()
{
    if (TryGetChange(_pathToElement, out change))
    {
        return change.Value;
    }

    return _originalJsonElement.GetString();
}

This diagram illustrates the layered design of DynamicData. MutableJsonDocument adds a layer of mutability around JsonDocument by storing changes made to the JSON in a change list. DynamicData adds a dynamic layer around MutableJsonDocument.

One of the characteristics we like in this design is that, if there aren’t any changes to the JSON, MutableJsonDocument behaves much the same as JsonDocument. And the implications regarding how you might use it, if they’re not already clear, are that making many changes to DynamicData might have a negative effect on performance. We’ve designed it to work best when you have a large JSON document and want to modify just a few properties before sending it back to the service—the “round-trip scenario” we described earlier. Authoring JSON from scratch, while possible, isn’t what we originally designed the type to do, and from an Azure SDK guidance perspective, we still recommend using anonymous types with RequestContent.Create for that.

What else does it do?

Because ToDynamicFromJson returns dynamic, you can’t inspect its public APIs using IntelliSense while you’re writing your code. To see what properties are available in the JSON content, you can view it in the Visual Studio debugger. For example:

BinaryData jsonData = BinaryData.FromString("""
    {
        "value": "Cyan"
    }
    """);
dynamic content = jsonData.ToDynamicFromJson();

If you run this code snippet while debugging in Visual Studio, you’ll see the following content in the Locals window:

Expanding the “Dynamic View” node lists the members in the JSON object. You can use this view, alongside the REST API documentation for the service you’re working with, to understand what properties are available in the JSON content. The property names used in the “Dynamic View” reflect the naming convention used in the underlying JSON. Remember that you can indicate to ToDynamicFromJson what naming convention this is by passing a JsonPropertyNames enum value, and use C# conventions in your code.

There are a few more features that might not be readily apparent. For example, DynamicData supports using the foreach keyword to iterate over the elements of JSON arrays and properties of JSON objects:

Response response = client.GetWidget();
dynamic widget = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);

// JSON is `{ "details" : { "color" : "blue", "size" : "small" } }`
foreach (dynamic property in widget.Details)
{
    Console.WriteLine($"Widget has property {property.Name}='{property.Value}'.");
}

You can also treat JSON arrays like standard .NET arrays by using indexers to retrieve array elements and the Length property to get their length:

Response response = client.GetWidget();
dynamic widget = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);

// JSON is `{ "values" : [1, 2, 3] }`
for (int i = 0; i < widget.Values.Length; i++)
{
    Console.WriteLine($"Value at index {i}: '{widget.Values[i]}'.");
}

You can use indexers for property access too. For example, if your JSON property name contains characters that are invalid for property names in C#:

Response response = client.GetWidget();
dynamic widget = response.Content.ToDynamicFromJson();

/// JSON is `{ "$id" = "123" }`
string id = widget["$id"];

If, for some reason, you need to bypass property name transformations that happen as a result of passing propertyNameFormat to ToDynamicFromJson, you can also use property indexers. For example, if you’ve passed JsonPropertyNames.CamelCase to ToDynamicFromJson and you need to set a property that has pascal-cased name, you use a property indexer to access the property with its exact name:

Response response = client.GetWidget();
dynamic widget = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);

widget.details["IPAddress"] = "127.0.0.1";
// JSON is `{ "details" : { "IPAddress" : "127.0.0.1" } }`

Note, however, that if your JSON uses mixed-cased property names, we recommend not passing any value for propertyNameFormat, and using exact property names in your C# code.

Finally, you can cast dynamic content to any .NET type, as long as the JSON it holds can deserialize to that type using System.Text.Json.JsonSerializer defaults and you pass the appropriate JsonPropertyNames value:

Response response = client.GetWidget();
dynamic content = response.Content.ToDynamicFromJson(JsonPropertyNames.CamelCase);

// JSON is `{ "id" : "123", "name" : "Widget" }`
Widget widget = (Widget)content;

public class Widget
{
    public string Id { get; set; }
    public string Name { get; set; }
}

For more examples showing what you can do with dynamic content, see our getting started documentation.

What’s next?

In the shorter term, you can expect to see tighter integration with Azure SDK model types, and an expanded set of types that can be assigned to DynamicData properties. Our change list-based implementation makes it easy to generate the JSON for JSON Merge Patch . We expect APIs specific to those operations to be added soon. We’re also planning to invest in performance improvements, so keep an eye out for all of that.

We’re excited for the future that ToDynamicFromJson enables, and we hope you’ll help us make it great for your specific needs by reaching out with suggestions and feedback in our GitHub repo. We look forward to hearing from you about your experiences, and even more, to working with you to improve the Azure SDK!

Anne Thompson Principal Software Engineer, Azure SDK

Follow