Announcing 0.2.0-alpha2 preview of Windows Azure WebJobs SDK

We are releasing an update to Windows Azure WebJobs SDK introduced by Scott Hanselman here.

Download this release

You can download WebJobs SDK in a console application project from the NuGet gallery. You can install or update to these packages through NuGet gallery using the NuGet Package Manager Console, like this:

Install-Package Microsoft.WindowsAzure.Jobs -Pre
Install-Package Microsoft.WindowsAzure.Jobs.Host -Pre

What is WebJobs SDK?

The WebJobs feature of Windows Azure Web Sites provides an easy way for you to run programs such as services or background tasks in a Web Site. You can upload and run an executable file such as an .exe, .cmd, or .bat file to your web site. You can run these as triggered or continuous WebJobs. Without WebJobs SDK, connecting and running background task requires a lot of complex programming. The SDK provides a framework that lets you write a minimum amount of code to get common tasks done.

The WebJobs SDK has a binding and trigger system which works with Windows Azure Storage Blobs, Queues and Tables. The binding system makes it easy to write code that reads or writes Windows Azure Storage objects. The trigger system calls a function in your code whenever any new data is received in a queue or blob.

The WebJobs SDK includes the following components:

NuGet packages. The NuGet packages enable your code to use the WebJobs SDK binding and trigger system with Windows Azure Storage tables, blobs and queues.
Dashboard. Part of the WebJobs SDK is already installed in Windows Azure Web Sites and provides rich monitoring and diagnostics for the programs that you write by using the NuGet packages. You don’t have to write code to use these monitoring and diagnostics features.

Scenarios for WebJobs SDK

Here are some typical scenarios you can handle more easily with the Windows Azure WebJobs SDK:

Image processing or other CPU-intensive work. A common feature of web sites is the ability to upload images or videos. Often you want to manipulate the content after it’s uploaded
Other long-running tasks that you want to run in a background thread, such as sending emails. Until now you couldn’t do this in ASP.NET because IIS would recycle your app if your app was idle for some time. Now with AlwaysOn in Windows Azure Web Sites you can keep the web site from being recycled when the app is idle. AlwaysOn ensures that the site does not go to sleep, which means you can run long-running tasks or services using WebJobs and the WebJobs SDK.
Queue processing. A common way for a web frontend to communicate with a backend service is to use queues. When the web site needs to get work done, it pushes a message onto a queue. A backend service pulls messages from the queue and does the work. This is a common producer – consumer pattern.
RSS aggregation. If you have a site that maintains a list of RSS feeds, you could pull in all of the articles from the feeds in a background process.
File maintenance, such as aggregating or cleaning up log files. You might have log files being created by several sites and you want to do analysis on them. Or you might want to schedule a task to run weekly to clean up old log files.

Goals of the SDK

Provide a way to make it easier to use Windows Azure Storage when doing any background processing work.
- The SDK makes it easier to consume Azure Storage within your application. You do not have to deal with writing code to read/ write from storage.
Provider a rich diagnostics and monitoring experience without having the user write any diagnostics and logging code.

Features of the SDK

Azure Storage

The SDK works with Azure Blobs, Queues and Tables.

Triggers

Functions get executed when a new input is detected on a Queue or a Blob. For example. In the following code ProcessQueue function will be triggered when a new message comes on a queue called “longqueue”. For more details on triggers please see this post.

Code Snippet

publicstaticvoid ProcessQueue([QueueInput(“longqueue”)] string output)
{
    Console.WriteLine(output);
 
}

Bindings

The SDK supports binding to provides model binding between C# primitive types and Azure storage like Blobs, Tables, and Queues. This makes it easy for a developer to read/ write from Blobs, Tables and Queues as they do not have to learn about the code around reading/ writing from Azure Storage.

Convenience. You can pick the type that’s most useful for you to consume and the WebJobs SDK will take care of the glue code. If you’re doing string operations on a blob, you can bind directly to TextReader/TextWriter, rather than worry about how to convert to a TextWriter.
Flushing and Closing: The WebJobs SDK will automatically flush and close outstanding outputs.
Unit testability. The SDK makes it possible to unit test your code since you can mock primitive types like TextWriter rather than ICloudBlob.
Diagnostics. Model binding works with the dashboard to give you real time diagnostics on your parameter usage.

The following Bindings are currently supported: Stream, TextReader/Writer, and String. You can add support for binding to your custom types and other types from the Storage SDK as well.

For more details on how Bindings work against Azure Storage, please read Blobs, Queues and Tables

Hosting

A JobHost is an execution container which knows what all functions do you have in your program. A JobHost object (which lives in Microsoft.WindowsAzure.Jobs.Host ) reads the bindings, listens on the triggers, and invokes the functions. In the following example, you create an instance of JobHost and call RunAndBlock(), which will cause the JobHost to listen for any triggers on any functions that you define in this Host.

Code Snippet

staticvoid Main(string[] args)
        {
            JobHost host = newJobHost();
            host.RunAndBlock();
 
        }

Dashboard for monitoring WebJobs.

As WebJobs (written in any language and of any type) execute, you can monitor them in real time. You can see their state (Running, Stopped, Successfully completed), last run time and the logs of a particular execution. The following screenshot shows you a view of all WebJobs running in your Website.

When you write a WebJob using the SDK, you get diagnostics and monitoring experience for the functions in your program. For example, let’s say that you have an Image processing WebJob called “ImageResizeAndWaterMark” that has the following flow.

When a user uploads an image to a Blob container called “images1-input”, the SDK will trigger WaterMark function. Watermark will process the image and write to “images2-input” container which will trigger the Resize function. Resize function will resize the image and write it to “images2-output” Blob container. The following code shows the WebJob described above. For a full working sample, please see the sample here

When you run the WebJob in Azure, you can view the WebJobs Dashboard by clicking the logs link of the “ImageResizeAndWaterMark” in the WEBJOBS tab of Windows Azure Websites portal.

Since the Dashboard is a SiteExtension you can access it by going to the url: https://mysite.scm.azurewebsites.net/azurejobs. You will need your deployment credentials to access the SiteExtension. For more information on accessing Site Extension, see the documentation on the Kudu project https://github.com/projectkudu/kudu/wiki/Accessing-the-kudu-service

Function execution details

When you are monitoring a particular execution of this “ImageResizeAndWaterMark” WebJob, you can view invocation details about the functions in the program such as:

What are the parameters of this function?
How much time did it take for the function to execute.
How much time did it take to read from Blob and how many bytes were read/ written.

Invoke & Replay

In the above example if the WaterMark function fails for some reason, you can upload a new image and Replay WaterMark function, which will trigger the execution chain and call Resize function as well. This is useful to diagnose and debug an issue when you have a complicated graph for chaining functions together. You can also Invoke a function from the dashboard.

Causality of functions

In the above example, we know that when the WaterMark function writes to a Blob, it will trigger the Resize function. The dashboard will show this causality between functions. If you have chained lots of functions which will get triggered as new inputs are detected then it can be useful to see this causality graph.

Search Blobs

You can click on Search for a Blob and get information on what happened to that Blob. For example, in the case of the ImageResizeAndWaterMark, the Blob was written because the WaterMark function got executed. For more details on Search Blobs see this post.

Samples

Samples for WebJobs SDK can be found at https://aspnet.codeplex.com/SourceControl/latest#Samples/AzureWebJobs/ReadMe.txt

You can find samples on how to use triggers and bindings for Blobs, Tables and Queues.
There is a sample called PhluffySuffy which is an Image processing Website where a customer can upload pictures which will trigger a function to process those pictures from Blob storage.

Documentation

Deploying WebJobs with SDK

If you don’t want to use the WebJobs portal page to upload your scripts, you can use FTP, git, or Web Deploy. For more information, see How to deploy Windows Azure WebJobs and Git deploying a .NET console app to Azure using WebJobs

If you want to deploy your WebJobs along with your Websites, check out the following Visual Studio extension.

Known Issues from 0.1.0-alpha1 to 0.2.0-alpha2

Dashboard will only work for WebJobs deployed with 0.2.0-alpha2

If you had a WebJob deployed with 0.1.0-alpha1 of SDK and, if you access the dashboard to see the logs for the WebJob, then you will see a warning about “Host not running”. This happens because as part of this release a newer version of the dashboard gets deployed to all Azure Websites. The new dashboard has some protocol changes which are not compatible with 0.1.0-alpha1. To work around this error, please update your WebJob to use 0.2.0-alpha2 NuGet package and redeploy your WebJob.

Give feedback and get help

The WebJobs feature of Windows Azure Web Sites and the Windows Azure WebJobs SDK are in preview and not formally supported. Feedback will be considered in changes we make to future versions.

If you have questions that are not directly related to the tutorial, you can post them to the Windows Azure forum, the ASP.NET forum, or StackOverflow.com. Use #AzureWebJobs for Twitter and the tag Azure-WebJobsSDK for StackOverflow.