The value of Cadl in designing APIs

The value of Cadl in designing APIs

As a developer, you want APIs that are well designed, intuitive, and easy to use. But you may encounter APIs that fall short on one or more of these dimensions, particularly in applications that combine data and services from multiple sources. The Azure REST API Guidelines establish the design principles and API patterns to ensure that Azure APIs are well designed, intuitive, and easy to use. But ensuring that service teams adhere to these guidelines becomes challenging to apply as the number of Azure services continues to expand, teams become more distributed, and our API surface area grows.

One challenge that we face is driving consistency in how our API definitions are written. Often, the API definitions are built by different individuals and teams within the same organization, and over time, can become large and complex. For example, the Azure Cognitive Services Metrics Advisor REST API definition is almost 7,000 lines! In addition to consistency and size, we also struggle with reuse. While reuse is relatively easy in code, it’s challenging when working with OpenAPI documents.

And Microsoft isn’t alone in these challenges. Many organizations are struggling through finding answers for how to build and manage APIs at scale. We’re seeing an emerging set of companies and tools that are working to provide solutions to difficult problems. One tool that Microsoft is beginning to have success with is Cadl, a new language for describing APIs.

The Cadl value proposition

Unlike other tools in the API space, Cadl is a language with a syntax that is similar to TypeScript, but built explicitly for describing APIs. In fact, we sometimes describe Cadl as “TypeScript for OpenAPI”. Because Cadl is a language, it provides many benefits that other tools don’t.

• Familiar syntax: Cadl borrows many language elements from popular programming languages like TypeScript and C#, so it’s easy for developers to learn and use.
• Great tooling support: IDE extensions in VS Code and Visual Studio include syntax highlighting, go to definition, linting, and auto-completion.
• Extensible: Developers can codify their API guidelines and best practices into a set of rules that can be enforced by Cadl.
• Highly expressive: API descriptions are concise and core concepts encapsulated into common libraries.

Authoring your API, you can compile your Cadl file to emit a complete OpenAPI definition that can be used to drive your existing tool chain. Cadl ensures that your API definition is correct and accurate. Just like you can’t compile a C# class with errors, you can’t produce an OpenAPI definition that doesn’t adhere to your rules and guidelines. By using Cadl, all of your APIs will be “correct by construction.”

Create an API design with Cadl

The code for this example is located at cadl-demo, and originated from the Http example in the Cadl-playground. It will help accelerate your understanding of this post if you load up this example and get familiar with Cadl. Cadl has rich tooling support in VS Code and Visual Studio, which gives developers full language support and makes refactoring APIs super easy. We’ll use VS Code and the OpenAPI extension for the rest of this post. We’ll also use examples from the Azure REST API Guidelines to illustrate how you can codify your own API standards.

Get started

Open the Cadl docs and select the Installation tab and then follow the steps there to install the tools and create a new Cadl project. The new empty Cadl project should contain:

package.json      # Package manifest defining your cadl project as a node package.
cadl-project.yaml # Cadl project configuration letting you configure emitters, emitter options, compiler options, etc.
main.cadl         # Cadl entrypoint

Next, we’ll create our initial Cadl file using the Cadl playground. The Cadl playground is a great place to experiment with and share simple Cadl definitions. Open the playground and select the Http service example. View the Swagger UI by choosing it from the dropdown in the right pane. We’ll make a few small changes that will help illustrate some points later in the discussion:

• Add version: 1.0.0 to the service decorator
• Change @key to @path on the id property in the Widgets model
• Add @route("/widgets") and @tag("Widgets") to the Widgets interface
• Remove @route from read
• Replace the @get customRead operation with @route("analyze") @post analyze() that returns string | Error;

The result should look like this:

import "@cadl-lang/rest";

@service({
  title: "Widget Service",
  version: "1.0.0"
})
namespace DemoService;
using Cadl.Http;

model Widget {
  @path id: string;
  weight: int32;
  color: "red" | "blue";
}

@error
model Error {
  code: int32;
  message: string;
}

@route("/widgets")
@tag("Widgets")
interface Widgets {
  @get list(): Widget[] | Error;
  @get read(@path id: string): Widget | Error;
  @post create(...Widget): Widget | Error;
  @route("analyze") @post analyze(): string | Error;
}

Copy the Cadl from the playground to the main.cadl file in your VS Code project. Now you can run cadl compile . in your VS Code terminal and it will produce an openapi.json file in the cadl-output directory with your generated OpenAPI 3 definition. If you open the openapi.json in VS Code and then open the Swagger UI view, it should look exactly like the view in the Cadl playground.

Add gadgets

Most services will have more than one resource (model) and endpoint. In this example, we’ll add the Gadget resource to our API definition. Using Cadl, it’s easy to add another model:

model Gadget {
  @path id: string;
  height: float32;
  width: float32;
  color: "green" | "yellow";
}

Now we want to add an endpoint for our new Gadget resource. We could copy and paste the Widgets interface, but then, we’d have to copy/paste the same operation definition for all endpoints! As developers, we prefer a way to define something once, and reuse it wherever we need it. Cadl supports this type of reuse for most API elements.

Build a Cadl library

Let’s use the power of the Cadl language to define an interface template that can be reused across all of our resources. We’ll create a new library.cadl file to contain this template and any common definitions. When a developer defines and new resource, they can define all the standard operations for the resource by extending the resource template interface. In this way, when the Cadl file is compiled, and the OpenAPI document emitted, it will have the required operations, constructed in exactly the way we defined them.

We’ll move the Error model from the main.cadl to the new file and then define an interface template that defines a list, create, and get operation similar to our Widgets interface, using a type parameter in place of the Widget model. The library.cadl now looks like:

import "@cadl-lang/rest";

using Cadl.Http;

@error
model Error {
  code: int32;
  message: string;
}

interface ResourceInterface<T> {
    @get list(): T[] | Error;
    @get read(@path id: string): T | Error;
    @post create(...T): T | Error;
  }

Back in the main.cadl file, we can now use the new ResourceInterface template for both Widgets and Gadgets as follows:

@route("/widgets")
@tag("Widgets")
interface Widgets extends ResourceInterface<Widget> {
  @route("analyze") @post analyze(): string | Error;
}

@route("/gadgets")
@tag("Gadgets")
interface Gadgets extends ResourceInterface<Gadget> {
}

Notice that we kept the analyze operation of Widgets because it isn’t part of the “standard” interface for all resources.

At this point, it’s worth noting that our Cadl definition is a mere 51 lines across two files but produces an OpenAPI definition that is 470 lines—an order of magnitude difference.

Integrate API guidelines

Next let’s look at how Cadl allows you to build API guidelines into your Cadl library. Organizations often create API guidelines to establish patterns for APIs that are well-designed, intuitive, and easy to use. Here, we’ll use Azure’s REST API guidelines as an example.

Error response

Azure’s API guidelines require an operation error response to have a standard structure and contents, in particular:

✅ DO return an x-ms-error-code response header with a string error code indicating what went wrong.

✅ DO provide a response body with the following structure

The error response in our current Cadl definition doesn’t meet these requirements, but that is easily remedied with a few changes to the error model in library.cadl. Since all operations are already using this one error model, we only need to make the change in this one place. Here’s the new definition that complies with the above guidelines:

@error
@doc("Error response")
model Error {
    @doc("A server-defined code that uniquely identifies the error.")
    @header("x-ms-error-code")
    code: string;
    @doc("Top-level error object")
    error: ErrorDetail;
}

@doc("Error details")
model ErrorDetail {
  @doc("A server-defined code that uniquely identifies the error.")
  code: string;
  @doc("A human-readable representation of the error.")
  message: string;
  @doc("An array of details about specific errors that led to this reported error.")
  details?: ErrorDetail[];
}

We’ve also added @doc decorators on the models and properties to produce descriptions on these elements in the generated OpenAPI, which will be shown in the REST API documentation generated from the OpenAPI.

Standard operations

Next, we’ll tackle the standard operations in our ResourceInterface template. The Azure API Guidelines require the response of a list operation to have a specific structure that supports extension and pagination (where needed). Specifically:

✅ DO structure the response to a list operation as an object with a top-level array field containing the set (or subset) of resources.

✅ DO return a nextLink field with an absolute URL that the client can GET in order to retrieve the next page of the collection.

☑️ YOU SHOULD use value as the name of the top-level array field unless a more appropriate name is available.

Azure also requires all operations to be idempotent, including creates. Most resources also need to support an update operation and a delete operation. Azure’s specific guidelines are:

✅ DO ensure that all HTTP methods are idempotent.

☑️ YOU SHOULD use PUT or PATCH to create a resource. These HTTP methods are easy to implement, allow the customer to name their own resource, and are idempotent.

✅ DO create and update resources using PATCH [RFC5789] with JSON Merge Patch (RFC7396) request body.

✅ DO return a 204-No Content without a resource/body for a DELETE operation.

Once again, we can comply with all the above guidelines with some straightforward changes to the library.cadl file:

• Define a model for the response structure of a list operation.
• Define a model for the JSON Merge Patch request body.
• Update the list operation to use the new response model.
• Change the create operation to use the put method.
• Add an update operation that accepts the new Patch model.
• Add a delete operation that returns void, which becomes a 204 response.

Here’s what the ResourceInterface looks like in our library.cadl file. Notice that it has all the standard operations you would expect. GET operations for the collection, a GET with an id for a specific resource, and the create, update, and delete operations with PUT, PATCH, and DELETE respectively. Here’s the changed section of library.cadl:

model List<T> {
    @doc("List of elements")
    value: T[];
    @doc("A link to the next page of results if present.")
    nextLink?: url;
}

model Patch<T> {
    @header contentType: "application/merge-patch+json";
    ...T;
}

interface ResourceInterface<T> {
    @get list(): List<T> | Error;
    @get read(@path id: string): T | Error;
    @put create(...T): T | Error;
    @patch update(...Patch<T>): T | Error;
    @delete delete(@path id: string): void | Error;
}

API versioning

Change is inevitable in all things including APIs, and Azure requires all APIs to implement explicit versioning support in the form of a required api-version query parameter on every operation. Here are some of the specific requirements from the Azure API Guidelines.

✅ DO use a required query parameter named api-version on every operation for the client to specify the API version.

✅ DO use “YYYY-MM-DD” date values, with a -preview suffix for preview versions, as the valid values for api-version.

Since the api-version query parameter must be passed on all operations, it makes sense to define it once in the library.cadl file. Here’s what that might look like:

model ApiVersion {
    @doc("The version of the API in the form YYYY-MM-DD")
    @query("api-version") apiVersion: string;
}

Then we add the apiVersion parameter to each operation in the ResourceInterface template using the spread operator on the ApiVersion model. Changing the template ensures that all the Widgets and Gadgets operations that derive from the template come into compliance with the above guidelines.

interface ResourceInterface<T> {
    @get list(...ApiVersion): List<T> | Error;
    @get read(@path id: string, ...ApiVersion): T | Error;
    @put create(...T, ...ApiVersion): T | Error;
    @patch update(...Patch<T>, ...ApiVersion): T | Error;
    @delete delete(@path id: string, ...ApiVersion): void | Error;
}

But we’re not yet fully in compliance with the API versioning requirements, because there’s still the analyze operation of the Widgets interface over in main.cadl that doesn’t have an api-version parameter. And as the API continues to grow, there may be other operations outside library.cadl that we need to make sure include an api-version parameter. Cadl has a solution for this problem: built-in linter support.

Lint with Cadl

Cadl allows libraries to define a function that checks a Cadl program for specific patterns and issue diagnostics (linter messages). Typically, these linter functions are written in TypeScript and then compiled and run as JavaScript as part of the Cadl compilation. For expediency, we’ll implement our linter in JavaScript directly, but we don’t endorse this approach for a production Cadl library.

We’ll implement our linter in a file called linter.js, which we import into library.cadl to activate it. Our linter will use functions from the @cadl-lang/lint library so we need to install that library in our project. You can see the full source code for linter.js in the cadl-demo project. Here are a few key points to focus on:

• Lines 9-15 define the version-policy diagnostic, which will be issued when the linter finds an operation without an api-version query parameter.
• Lines 20-22 define a function that checks if a parameter has the necessary attributes of an api-version parameter–its name, it’s required (not optional), and it’s a string type.
• Lines 25-40 define the versionPolicyRule, which checks each operation (line 26) to see if “some” parameter is an apiVersion parameter and a query parameter. If not, the diagnostic is issued.
• Lines 46-47 register and enable the rule.

With the linter now included in our library, cadl compile . will issue a diagnostic and fail the compile if any operation is missing the api-version query parameter:

>cadl compile .
Cadl compiler v0.38.3

/Users/mikekistler/Projects/mikekistler/cadl-demo/main.cadl:27:3 - error myLibrary/version-policy: Every operation must have a required 'api-version' query parameter
> 27 |   @route("analyze") @post analyze(): string | Error;
     |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Found 1 error.

Summary

In this post, we’ve taken you through some of the key value propositions of Cadl:

• Cadl provides a natural and concise expression of API constructs that produces a standard OpenAPI v3 API definition.
• Cadl enables definition and reuse of API patterns.
• Cadl makes it easy to incorporate your organization’s API guidelines into these reusable patterns so that when developers use them they automatically get compliant APIs.
• Cadl allows you to build linters right into your reusable libraries that signal to developers when they’ve possibly crossed one of your guidelines.

All these features add up to a system that enables developers to build APIs that are well designed, intuitive, and easy to use.

There’s much more to share on Cadl, such as how to create standalone libraries that can be reused across many projects, advanced API patterns, creating customer emitters, and more. Look for more posts on Cadl coming soon!

Cadl