• Dev Blogs
  • Azure SDK Blog
  • Process Azure Digital Twins data updates with the new Azure Function Extension Libraries **Beta**
July 29th, 2021

Process Azure Digital Twins data updates with the new Azure Function Extension Libraries **Beta**

Muhammad Nabeel
Muhammad Nabeel
Senior Customer Engineer

Introduction

Hello, my name is Muhammad Nabeel and I’m a Senior Customer Engineer at Microsoft. I help customers in their journey to use the Azure platform to build solutions. In a previous post, we introduced the new Azure Function Extension libraries for Storage, Event Hubs, Service Bus, and Event Grid. Now with Azure SDK’s Azure Function Extension Libraries (Beta) we can build truly secretless systems. The new extensions make it possible to use secretless configuration and authenticate via Azure Active Directory. These Azure platform capabilities eliminate the responsibility of managing and maintaining secrets for various services.

In this blog, I will demonstrate the new Azure Function Extension Libraries (Beta) for a secretless configuration of an IoT system. This configuration shifts the responsibility of managing secrets from an Azure Function operator to the Azure platform using Azure Active Directory. This post will show how data is flowing from Azure Digital Twins to Azure Event Hubs and consumed by an Azure Function with using secrets stored in configuration.

Here is a summary:

  1. Set up Azure Digital Twins to route events to Event Hubs
  2. Set up Azure Function App to process Event Hubs Trigger
  3. Add code to Azure Function App to capture the Azure Digital Twins events coming through Event Hubs Trigger

Prerequisites

Architecture overview

Data flow

There are two main components of the architecture:

  • The Azure Digital Twins sending messages to Event Hubs.

  • The Azure Function consuming the Event Hub messages using secretless configuration.

The underlying core of the architecture is the use of the beta version for Event Hubs Functions extensions that provides us to use Consume EventHub messages without having to use any secrets.

Set up Azure Digital Twins to route events to event hub

Routing events from Azure Digital Twins to Azure Event hub is a simple two-step process.

Create Azure Digital Twins Endpoint

Following image illustrates how to create an Azure Digital Twins Endpoint:

Create endpoint

Create Azure Digital Twins Event Route

Following image illustrates how to create Azure Digital Twins Event Route:

Create event route

Set up Azure Function App to process Event Hub Trigger

Following are the configurations that will enable secretless configuration.

Enabled Managed Identity for the Function App

To enable secretless configuration, we will use Azure Active Directory to authenticate Azure Function App. The first step in this process is to enable Managed Identity for the function app.

Here are the steps:

  1. Sign in to the Azure portal and navigate to your Azure Function App that was mentioned in the prerequisite section.
  2. On the left navigation menu select “Identity”.
  3. On the “System assigned” tab, select “On” toggle button under “Status” heading.
  4. Select “Save”.

Following image illustrates the steps:

Enabled Managed Identity for the Function App

Add application setting for EventHub connection

When we use Azure Function to listen to EventHub triggers, we add EventHubConnection as an application setting. Here is an example of such EventHubConnection setting:

"EventHubConnection": "Endpoint=sb://santacruz3203-digitaltwin-ehnamespace.servicebus.windows.net/;SharedAccessKeyName=EHPolicy;SharedAccessKey=Your-Shared-Access-Key=;EntityPath=santacruz3203-digitaltwin-eventhub"

You can see in above image, secret is included as part of the EventHubConnection setting.

The secretless configuration takes away the use of secrets in the configuration. So the above mentioned connection string will now look like this:

"EventHubConnection__fullyQualifiedNamespace": "santacruz3203-digitaltwin-ehnamespace.servicebus.windows.net"

There are two things worth noting from the above configuration setting:

  1. There is no secret mentioned in the Application Setting for EventHubConnection.
  2. Addition of “__fullyQualifiedNamespace” as part Application setting name. This change is how the beta version for Storage, Event Hubs, Service Bus, and Event Grid Functions extensions know to use secretless configuration.

Here are the steps to accomplish this change:

  1. Sign in to the Azure portal and navigate to Azure Function App that was mentioned in the prerequisite section.

  2. On the left navigation menu click “Configuration”.

  3. On the “Application settings” tab, click “+ New application setting” button under “Application settings” heading.

  4. Add “EventHubConnection__fullyQualifiedNamespace” as application setting name.

  5. Add fully qualified namespace as application setting value. You can retrieve fully qualified namespace by opening Event Hub Namespace on Azure portal and retrieving the given for “Host name” as shown below: Retrieve fully qualified namespace

  6. Select “OK”.

  7. Select “Save”.

Following image illustrates the above mentioned steps:

Enabled Managed Identity for the Function App

Add role assignment for Azure Function in Event Hub instance

The final part of the Azure Function configuration is enabling managed identity for Azure Function in EventHub.

Here are the steps:

  1. Sign in to Azure portal and navigate to the EventHub that was mentioned in the prerequisite section. This will open EventHub instance controls.
  2. On the left navigation menu select “Access control (IAM)”.
  3. Select “+ Add”.
  4. Choose “Add role assignment”.
  5. On the “Add role assignment” panel, select:
    1. “Azure Event Hubs Data Receiver” as role.
    2. “Function App” for “Assign access to” pull-down menu.
    3. For subscription, select the subscription where the Azure Function belongs to.
    4. Select the Azure Function.
  6. Select “Save”.

Following image illustrates the above mentioned steps:

Add role assignment for Azure Function in Event Hub instance

Add code to Azure Function App

From a coding perspective, nothing changes when using secretless configuration. The Azure Function code will be written exactly the same way as for configuration with secrets. So if you have already written Azure Function having configuration with secrets, then you would not need to change the actual code. The only change will be on the configuration side. This has already been mentioned under the section “Set up Azure Function App to process Event Hub Trigger”.

Besides the configuration, we have to ensure we are using the Azure Function Extension Libraries (Beta). For the prerelease version of “Microsoft Azure WebJobs SDK EventHubs Extension“, the NuGet package should be added to Azure Function project.

Following Package Manager command can be used to install the package:

Install-Package Microsoft.Azure.WebJobs.Extensions.EventHubs -Version 5.0.0-beta.6

If Visual Studio is being used to develop Azure Function, then prerelease version of “Microsoft Azure WebJobs SDK EventHubs Extension” library should be added to Azure Function project as shown below:

Package setup

Complete code can be found at aka.ms/AAd4roz

Here is how the code looks like:


using Azure.Messaging.EventHubs;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using TwinsUpdateBetaFunctionApp.Model;

namespace TwinsUpdateBetaFunctionApp
{
    public static class TwinsUpdateBetaFunction
    {
        [FunctionName("TwinsUpdateBetaFunction")]
        public static void Run([EventHubTrigger("santacruz3203-digitaltwin-eventhub", Connection = "EventHubConnection")] EventData[] events, ILogger log)
        {
            var exceptions = new List<Exception>();
            foreach (EventData eventData in events)
            {
                try
                {
                    log.LogInformation($"Message received: {eventData.EventBody}");

                    // Deserialize the message into TwinMessage object. TwinMessage is a custom class representing the schema of Azure Digital Twins message.
                    TwinMessage twinMessage = eventData.EventBody.ToObjectFromJson<TwinMessage>();

                    // Here add your code to further process TwinMessage.

                    log.LogInformation($"Message processed");
                }
                catch (Exception e)
                {
                    exceptions.Add(e);
                }
            }

            if (exceptions.Count > 1)
                throw new AggregateException(exceptions);

            if (exceptions.Count == 1)
                throw exceptions.Single();
        }
    }
}

Let us take a look at the code.

  • The flow starts when event hub triggers this function.
  • On the trigger we get a list of events.
  • For each event date (part of events list), we are reading the message body.
  • Each message body is deserialized to get jToken for each data field.

The Azure Digital Twins, Update Function App, stops the flow to create an opening to bridge with other down stream systems. One example could be a frontend platform that shows the Azure Digital Twins updates data on a view.

Conclusion

Azure SDK is adding capabilities to allow secretless configuration implementation. This is huge in terms of management of secrets. With Azure Function Extension Libraries (Beta), cloud architectures can be designed that use Azure AD for authentication without the need of maintaining secrets. In the post, we learned that it is possible to build an IoT system without secrets.

Azure SDK Releases

Azure SDK Blog Contributions

Thank you for reading this Azure SDK blog post! We hope that you learned something new and welcome you to share this post. We are open to Azure SDK blog contributions. Contact us at azsdkblog@microsoft.com with your idea and we’ll get you set up as a guest blogger.

Azure SDK Links

Category
Azure SDK
Topics
azureAzure Digital TwinsAzure Event HubAzure FunctionAzure Function Extensionazure-sdkEventHubIoTsecrets

Author

Muhammad Nabeel
Muhammad Nabeel
Senior Customer Engineer

I’m a Senior Customer Engineer at Microsoft. I help customers in their journey to use the Azure platform to build solutions. I am based out of Orange County, CA. I love hiking and biking with my kids.

0 comments

Discussion are closed.

Read next