Have you ever needed to just run your Java code in an instant-on container in the cloud without the hassle of managing your own Kubernetes server?  How about having access to Open AI code interpreter built-in to that container and billed on a consumption basis?

About Azure Container Apps and dynamic sessions

Azure Container Apps (ACA) dynamic sessions provide fast access to secure, sandboxed environments that have access to Azure Open AI (AOAI) via a code interpreter.  ACA dynamic sessions offer strong isolation, access through a REST API, and are automatically deleted when no longer in use. The fast startup times achieved by maintaining a pool of ready sessions in ACA make it possible to allocate new sessions in milliseconds. dynamic sessions are also highly scalable, capable of running hundreds or thousands of sessions concurrently.

Code Interpreter and Java

Code interpreter sessions run code via code execution or uploaded files in a sandbox preinstalled with popular libraries, making them ideal for executing untrusted code provided by users or generated by large language models (LLMs).  The default language for the container is Python, but most languages, including Java, are supported by the code interpreter.  Learn more about ACA dynamic sessions on Microsoft Learn.

POC Repo – aca-dynamic-sessions (github.com)

I’ve created a simple POC to explore how to use ACA dynamic sessions with Java via HTTP.   Stay tuned for a version that uses LangChain4J!  in the meantime, here’s the simple POC on GitHub.  Check out the readme for full details on prerequisites and setup.

For simplicity I’ve built the POC as a Spring Boot application that communicates with the dynamic sessions API, and covers the basic interactions with dynamic sessions, including executing code, plus uploading, listing and downloading files.

Main Application Code The main class, ACADynamicSessions, sets up the application and coordinates calls to other classes that perform various tasks. It retrieves variables defined in an application.properties file, then calls the getToken method to get an access token from Azure.

The getToken method uses the Azure Identity library to get an access token. This token is used to authenticate the application’s requests to the “dynamicsessions” service. The token is requested for the scope “https://dynamicsessions.io/.default”, which means it’s requested for all available permissions on the “dynamicsessions” service.

The getToken() Method is used to obtain an access token from Azure Active Directory (AAD) using the Azure Identity library.

private static String getToken() {
        DefaultAzureCredential credential = new DefaultAzureCredentialBuilder().build();
        TokenRequestContext requestContext = new TokenRequestContext().addScopes("https://dynamicsessions.io/.default");
        AccessToken token = credential.getToken(requestContext).block();
        return token.getToken();
    }

Here’s a breakdown of what this code does:

1. Create a DefaultAzureCredentialobject. This object represents a credential capable of providing an AAD token from multiple sources, such as environment variables, managed identity, or a local development service like Azure CLI or Visual Studio Code.
2. Create a TokenRequestContext object and add a scope to it. The scope is a string that represents the resource and permissions your application is requesting access to. In this case, the scope is “https://dynamicsessions.io/.default”, which represents the default permissions for the dynamic sessions API.
3. The block() method is called to requests the access token synchronously, meaning the method will not return until it has either successfully obtained a token or encountered an error.
4. Return the actual string representation of the access token.

This access token can then be used to authenticate requests to the dynamic sessions API.

• ACADynamicSessions also:
• Constructs several URLs using the retrieved configuration values and the access token.
• Calls methods from ExecuteCode, UploadFile, ListFiles, and DownloadFile classes, passing the constructed URLs and the access token to them.

Code Execution

ExecuteCode contains a single static method execute(String url, String token).

The execute method is designed to send an HTTP POST request via the OkHttpClient object to handle the HTTP request.to a specified URL with a JSON payload. The JSON payload contains properties for executing inline Python code (print('Hello, world!'))  synchronously.  The client sends the request and gets a Response object. The response body is printed to the console.

This code executes Python and returns a response that can be printed to the console, just to confirm that you have access to the Dynamic Session.  The default language for the container is Python, but Java can then be executed via the code interpreter inside the Python container.

 String json = "{"
        + "\"properties\": {"
        + "\"codeInputType\": \"inline\","
        + "\"executionType\": \"synchronous\","
        + "\"code\": \"print('Hello, world!')\""
        +  "}"
        +  "}";

        RequestBody body = RequestBody.create(json, MediaType.parse("application/json"));
        Request request = new Request.Builder()
            .url(url)
            .post(body)
            .addHeader("Authorization", "Bearer " + token)
            .addHeader("Content-Type", "application/json")
            .build();

Working with Files

The UploadFile class is used to upload a file to dynamic sessions using the OkHttp library.  It takes three parameters: url (the server URL to which the file will be uploaded), token (the authorization token for the server), and filename (the path of the file to be uploaded).  A File object is created with the provided filename.  A RequestBody is created from the file and the media type is set to “application/json” and a MultipartBody is created and set to the type FORM. The file data is added to this body with the form data part name “file”.  The response is captured in a Response object.

OkHttpClient client = new OkHttpClient();
            
        File file = new File(filename);
        RequestBody fileBody = RequestBody.create(file, MediaType.parse("application/json"));
        RequestBody body = new MultipartBody.Builder()
            .setType(MultipartBody.FORM)
            .addFormDataPart("file", file.getName(), fileBody)
            .build();

        Request request = new Request.Builder()
            .url(url)
            .post(body)
            .addHeader("Authorization", "Bearer " + token)
            .addHeader("Content-Type", "multipart/form-data")
            .build();

The ListFiles class sends a GET request to a specified URL, parses the JSON response, and return a list of filenames.  It takes two parameters: url (the server URL to which the file will be uploaded), and token (the authorization token for the server).   The response body is parsed as a JSON object, then a StringBuilder object is created to store the filenames.  The code loops through each object in the “value” array in the JSON Object, then the string representation of the StringBuilder (which contains all filenames separated by commas) is returned.

        OkHttpClient client = new OkHttpClient();

        Request request = new Request.Builder()
            .url(url)
            .get()
            .addHeader("Authorization", "Bearer " + token)
            .build();

        try (Response response = client.newCall(request).execute()) {
            if (!response.isSuccessful()) {
                throw new IOException("Unexpected code " + response);
            }

            // Parse the response body as JSON
            JSONObject json = new JSONObject(response.body().string());

            // Create a StringBuilder to store the filenames
            StringBuilder filenames = new StringBuilder();

            // Get the "value" array from the JSON object
            JSONArray valueArray = json.getJSONArray("value");

            // Loop through each object in the "value" array
            for (int i = 0; i < valueArray.length(); i++) {
                // Get the current object
                JSONObject currentObject = valueArray.getJSONObject(i);

                // Get the "properties" object from the current object
                JSONObject properties = currentObject.getJSONObject("properties");

                // Get the filename from the "properties" object
                String filename = properties.getString("filename");

                // Append the filename to the StringBuilder
                filenames.append(filename);

                // If this is not the last filename, append a comma and a space
                if (i < valueArray.length() - 1) {
                    filenames.append(", ");
                }
            }

            // Return the string representation of the StringBuilder
            return filenames.toString();

The DownloadFile class is a simple example of how to download a file in Java using a GET request to a specified URL. It takes two parameters: a url from which to download the file, which is built from the ListFIles response, and a token for authorization.  The body of the response is printed to the console, representing the content of the downloaded file.

OkHttpClient client = new OkHttpClient();

        Request request = new Request.Builder()
            .url(url)
            .get()
            .addHeader("Authorization", "Bearer " + token)
            .build();

Summary

Let me know if this inspires you to use Azure Container Apps dynamic sessions to run Java code and why!  Remember to check the overview of ACA dynamic sessions on Microsoft Learn and the full POC repo on GitHub at https://aka.ms/aca-dynamic-sessions.  And check back for a new version of this code using Langchain4J, coming soon!

Happy Coding!