Azure Durable Functions is a new programming model based on Microsoft serverless’ platform Azure Functions. It allows you to write a workflow as code and have the execution run with the scalability and the reliability of serverless with high throughput.<\/p>\n
Initially, I wanted to index data from GitHub repositories. I explained it all in a\u00a0previous post<\/a>, but I’ll drill down into the concepts today.<\/p>\n To summarize the post, I wanted to retrieve a list of repositories that I wanted to index. Then, query each repository for more detailed information. Finally, save the data into a database.<\/p>\n Today, we’ll do something similar but to keep things simple, I’ll save the content to Azure Blob Storage.<\/p>\n If you want to follow along, I’ve created\u00a0a sample<\/a>\u00a0that contains all the code in this post.<\/p>\n Let’s get started.<\/p>\n Besides the common\u00a0serverless concepts<\/a>, Azure Durable Functions introduces more concepts that make developing complex workflow easy.<\/p>\n Let’s start with the basic concepts and work our way up.<\/p>\n A serverless function on Azure is a unit of work that is, in C#, a single method from a static class. Azure Functions allows you only pay for what you use as well as scaling that piece of code for you.<\/p>\n Generous\u00a0free quotas<\/a>\u00a0are also given to allow you to experiment quickly and cheaply in the cloud.<\/p>\n In C#, we can write a simple Function with the following code.<\/p>\n A trigger is an event happening to which a piece of code responds.<\/p>\n It is part of what is called a\u00a0binding<\/em>\u00a0which hooks a method, a variable, or a parameter of your code to external systems.<\/p>\n Bindings come in 3 types, but we’re only interested in the latter. For more information on bindings, you can read the\u00a0concept page<\/a><\/p>\n By looking at the list of\u00a0supported bindings<\/a>, we see a\u00a0 The most common triggers, however, are\u00a0 Those triggers are applied directly on a parameter of a function.<\/p>\n Those are traditional functions that you find in a serverless application deployed on Azure. They are simple and respond to events.<\/p>\n What if you need something more complicated? What about calling functions with functions? For example, invoking one function for every element of a list then reconciling the result and saving it to a file.<\/p>\n Writing a complex workflow with simple code is where Durable Functions comes in.<\/p>\n An Orchestrator is an Azure Function with specific behaviors attached to them. As an example, it automatically set checkpoint during its execution, shut itself down while waiting for other functions to finish executing, then replay through the checkpoint to resume execution.<\/p>\n They are at the very heart of how Durable Functions works.<\/p>\n It is important to remember how an Orchestrator function operates differently than a standard function. While a normal function only executes once per event, the orchestrator is restarted many times while waiting for other functions to complete.<\/p>\n This kind of behavior means that this function needs to be\u00a0deterministic<\/em>. It must return the same result each time. It is crucial then to not use\u00a0 An Orchestrator function is a function with an\u00a0 It is invoked by a\u00a0 At this point, we have an orchestrator that is ready to run other functions.<\/p>\n Not every function can be used with an orchestrator. Those functions become an Activity Function when an\u00a0 Activities are part of the checkpoint and replay pattern described previously. Among the differences with other functions, they do not respond to other triggers than an orchestrator calling them.<\/p>\n An activity is a perfect place to have code accessing a database, sending emails, or any other external systems. Activities that have been called and returned have their results cached. Meaning, that returning 1000 rows from a database won’t be done multiple times but only a single time.<\/p>\n As an example, our sample uses an activity function to retrieve the list of repositories from which we want to extract more data.<\/p>\n An Activity Function is not re-executed for the same Orchestrator instance. In this scenario, we preserve the list of repositories without invoking the API again.<\/p>\n This caching mechanism gives us significant leverage while orchestrating complex workflows. First, it allows us to keep the list of data on which we operate fixed. Second, since the orchestrator may call that function multiple times, it’s essential to have the result cached to avoid crashing external systems. Finally, it becomes possible to debug what we send\/return to activities since every execution is logged transactionally by the Durable Functions component.<\/p>\n With those concepts clarified, let’s look at Code as Orchestration.<\/p>\n Workflows in the past were traditionally built with either a graphic designer or data persistence format (XML, JSON, YAML, or other). Those are easy to create and understand for small workflow. However, they become tough to debug and understand where a failure occurs.<\/p>\n On top of that, they were never meant to run on a high scale infrastructure where one part of the workflow would need to be scale differently.<\/p>\n Let’s take a simple example.<\/p>\n I want to retrieve the list of opened issues for all repositories within a GitHub organization and save the count somewhere in the cloud.<\/p>\n<\/blockquote>\n Here’s how it looks in pseudo-code.<\/p>\n Here’s how you would represent it with Azure Durable Functions.<\/p>\n It is the same code found in\u00a0the sample<\/a>\u00a0on GitHub.<\/p>\n Let’s walk through this orchestrator line by line. I retrieve an organization name from the function that triggered the Orchestrator. The input mechanism allows me to reuse the code for more than a single purpose. Then, I invoke a function that returns me a list of all repositories.<\/p>\n Once the list is returned, and not before, we invoke as many\u00a0 Finally, we wait for all the functions to complete their executions before sending the compiled results to a\u00a0 With as few as 8 lines of codes, we managed to build a complex workflow without XML\/JSON that your team can understand quickly.<\/p>\n Give it a try now by downloading\u00a0the sample<\/a>\u00a0and trying it out! If you want to run it in the cloud,\u00a0get a free Azure subscription<\/a>\u00a0and take advantage of free Azure Functions.<\/p>\n To get more information on Azure Durable Functions here’s some reading material.<\/p>\n Azure Durable Functions is a new programming model based on Microsoft serverless’ platform Azure Functions. It allows you to write a workflow as code and have the execution run with the scalability and the reliability of serverless with high throughput. Scenario Initially, I wanted to index data from GitHub repositories. I explained it all in […]<\/p>\n","protected":false},"author":41506,"featured_media":58792,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[685],"tags":[4,9,37],"class_list":["post-20635","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-dotnet","tag-net","tag-net-core","tag-azure"],"acf":[],"blog_post_summary":" Azure Durable Functions is a new programming model based on Microsoft serverless’ platform Azure Functions. It allows you to write a workflow as code and have the execution run with the scalability and the reliability of serverless with high throughput. Scenario Initially, I wanted to index data from GitHub repositories. I explained it all in […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/20635","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/users\/41506"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/comments?post=20635"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/posts\/20635\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media\/58792"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/media?parent=20635"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/categories?post=20635"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/dotnet\/wp-json\/wp\/v2\/tags?post=20635"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}Concepts<\/h2>\n
Definition of a Function<\/h3>\n
public static class MyFunction\r\n{\r\n [FunctionName(\"MyFirstFunction\")]\r\n public void MyFirstFunction()\r\n {\r\n \/\/...\r\n }\r\n}<\/code><\/pre>\nDefinition of a trigger<\/h3>\n
\n
Trigger<\/code>\u00a0column. Triggers include storage events (Blob Storage, Cosmos DB, or other.) as well as eventing systems such as Event Grid and Event Hubs.<\/p>\nHttp<\/code>, which respond to an HTTP request from a user, as well as\u00a0Timer<\/code>\u00a0which allows you to specific code on a specified schedule.<\/p>\npublic static class MyFunction\r\n{\r\n \/\/ The timer trigger is based on a CRON expression.\r\n \/\/ {second} {minute} {hour} {day} {month} {day-of-week}\r\n \/\/ In this case, the timer is every 5 minutes.\r\n [FunctionName(\"MyTimerTrigger\")]\r\n public static void MyTimerTrigger([TimerTrigger(\"0 *\/5 * * * *\")]TimerInfo myTimer) { \/* ... *\/ }\r\n\r\n [FunctionName(\"HttpTriggerCSharp\")]\r\n public static async Task Run(\r\n [HttpTrigger(AuthorizationLevel.Function, \"get\", \"post\", Route = null)]HttpRequestMessage req) { \/* ... *\/}\r\n}<\/code><\/pre>\nDefinition of an Orchestrator Function<\/h3>\n
DateTime.Now<\/code>,\u00a0Guid.NewGuid()<\/code>\u00a0or anything generating a different result in this method.\u00a0More information is available on the checkpoint and replay pattern that Orchestrators are using internally<\/a>.<\/p>\nOrchestrationTrigger<\/code>\u00a0on a\u00a0DurableOrchestrationContext<\/code>\u00a0object parameter.<\/p>\n[FunctionName(\"Orchestrator\")]\r\npublic static async Task RunOrchestrator([OrchestrationTrigger] DurableOrchestrationContext context)\r\n{\r\n return context.InstanceId;\r\n}<\/code><\/pre>\nDurableOrchestrationClient<\/code>\u00a0decorated with an\u00a0OrchestrationClientAttribute<\/code>\u00a0injected within an ordinary function with any triggers. Let’s take a\u00a0Timer<\/code>\u00a0trigger as an example.<\/p>\n[FunctionName(\"MyTimerTrigger\")]\r\npublic static void MyTimerTrigger([TimerTrigger(\"0 *\/5 * * * *\")]TimerInfo myTimer, [OrchestrationClient]DurableOrchestrationClient starter)\r\n{\r\n \/\/ Function input comes from the request content.\r\n string instanceId = await starter.StartNewAsync(\"Orchestrator\", null);\r\n \/* ... *\/\r\n}<\/code><\/pre>\nDefinition of an Activity Function<\/h3>\n
ActivityTriggerAttribute<\/code>\u00a0is on a\u00a0DurableActivityContext<\/code>\u00a0parameter.<\/p>\n[FunctionName(\"MyActivity\")]\r\npublic static async Task MyActivity([ActivityTrigger] DurableActivityContext context)\r\n{\r\n \/* ... *\/\r\n}<\/code><\/pre>\n[FunctionName(\"GetAllRepositoriesForOrganization\")]\r\npublic static async Task<List> GetAllRepositoriesForOrganization([ActivityTrigger] DurableActivityContext context)\r\n{\r\n \/\/ retrieves the organization name from the Orchestrator function\r\n var organizationName = context.GetInput();\r\n \/\/ invoke the API to retrieve the list of repositories of a specific organization\r\n var repositories = (await github.Repository.GetAllForOrg(organizationName)).Select(x => (x.Id, x.Name)).ToList();\r\n return repositories;\r\n}<\/code><\/pre>\nCode as orchestration<\/h3>\n
\n
repositories = getAllRepositories(organizationName);\r\nforeach( repository in repositories)\r\n repository.getOpenedIssuesCount()\r\ncloud.save(repository)<\/code><\/pre>\n[FunctionName(\"Orchestrator\")]\r\npublic static async Task RunOrchestrator(\r\n [OrchestrationTrigger] DurableOrchestrationContext context)\r\n{\r\n \/\/ retrieves the organization name from the Orchestrator_HttpStart function\r\n var organizationName = context.GetInput();\r\n \/\/ retrieves the list of repositories for an organization by invoking a separate Activity Function.\r\n var repositories = await context.CallActivityAsync<List>(\"GetAllRepositoriesForOrganization\", organizationName);\r\n\r\n \/\/ Creates an array of task to store the result of each functions\r\n var tasks = new Task[repositories.Count];\r\n for (int i = 0; i < repositories.Count; i++)\r\n {\r\n \/\/ Starting a `GetOpenedIssues` activity WITHOUT `async`\r\n \/\/ This will starts Activity Functions in parallel instead of sequentially.\r\n tasks[i] = context.CallActivityAsync(\"GetOpenedIssues\", (repositories[i]));\r\n }\r\n\r\n \/\/ Wait for all Activity Functions to complete execution\r\n await Task.WhenAll(tasks);\r\n\r\n \/\/ Retrieve the result of each Activity Function and return them in a list\r\n var openedIssues = tasks.Select(x => x.Result).ToList();\r\n\r\n \/\/ Send the list to an Activity Function to save them to Blob Storage.\r\n await context.CallActivityAsync(\"SaveRepositories\", openedIssues);\r\n\r\n return context.InstanceId;\r\n}<\/code><\/pre>\nGetOpenedIssues<\/code>\u00a0functions as necessary to complete our task. Note that we are not using the\u00a0await<\/code>\u00a0keyword here meaning that they are executing in parallel instead of sequentially.<\/p>\nSaveRepositories<\/code>\u00a0function to save them to Blob Storage.<\/p>\nResources<\/h2>\n
\n