QueueBackgroundWorkItem to reliably schedule and run background processes in ASP.NET

Stack Overflow is loaded with questions on how to reliably run a resource-intensive process on a background thread. See so0, so1, so2, so3, so4, so5, so6, so7, so8, so9, so10 . Examples of long-running tasks include sending email, image processing, and generating a PDF file. When Phil Haack was a program manager on the ASP.NET MVC team, he wrote the definitive blog on the inherent unreliability of running background tasks on ASP.NET. While Phil’s blog is a good read, there are now three supported approaches to launching long-running process on ASP.NET:

Cloud Services worker role is an environment in which you can run code. It’s basically a computer, really. You run whatever code you want (EXE, BAT, PS1, NodeJS, .NET, etc.) An Azure worker role provides the most industrial strength and scalable solution to this problem. For an excellent tutorial with this approach, see Tom Dykstra’s Get Started with Azure Cloud Services and ASP.NET.
WebJobs (including the WebJobs SDK) are a way in Azure to run scheduled tasks or tasks that trigger on demand (given various types of triggers). Apps specifically written with the Azure Jobs SDK can be used to run code in any environment, including a local computer, Azure Web Site, Azure Worker Role, Azure VM, etc. Although you can run them anywhere, they run most efficiently within Azure. For more information see Azure WebJobs – Recommended Resources.
QueueBackgroundWorkItem (QBWI). This was specifically added to enable ASP.NET apps to reliably run short-lived background tasks. (With some limitations explained at the end of this blog.) As of today, you can’t use QBWI on an Azure Web Site or Cloud Services web role because QBWI requires .Net 4.5.2. We hope to have Azure Web/Cloud running .Net 4.5.2 soon.

In addition to the three supported approaches above, the open source HangFire package allows you to run background tasks.

QueueBackgroundWorkItem overview

QBWI schedules a task which can run in the background, independent of any request. This differs from a normal ThreadPool work item in that ASP.NET automatically keeps track of how many work items registered through this API are currently running, and the ASP.NET runtime will try to delay AppDomain shutdown until these work items have finished executing.

QueueBackgroundWorkItem API

[SecurityPermission(SecurityAction.LinkDemand, Unrestricted =true)]
public static void QueueBackgroundWorkItem(Action<CancellationToken> workItem);

Takes a void-returning callback; the work item will be considered finished when the callback returns.

[SecurityPermission(SecurityAction.LinkDemand, Unrestricted = true)] 
public static void QueueBackgroundWorkItem(Func<CancellationToken, Task> workItem);

Takes a Task returning callback; the work item will be considered finished when the returned Task transitions to a terminal state.

Send email with attachment using QBWI

To use QBWI (QueueBackgroundWorkItem) in Visual Studio, you’ll need to install .Net 4.5.2, then install the .Net 4.5.2 Developer Pack. For my sample I created an MVC app and used SendGrid to send an email with a large jpg attachment. To use QBWI, you’ll need to right click the project in solution explore and select Properties. Select the Application tab on the left, then select .Net Framework 4.5.2 in the Target Framework dropdown. If you don’t see 4.5.2, you didn’t install the .Net 4.5.2 Developer Pack or you don’t have .Net 4.5.2 installed.

The following code sends email with an image file attached:

 public ActionResult SendEmail2([Bind(Include = "Name,Email")] User user)
 {
    if (ModelState.IsValid)
    {
       HostingEnvironment.QueueBackgroundWorkItem(ct => SendMailAsync(user.Email));
       return RedirectToAction("Index", "Home");
    }

    return View(user);
 }   

 private async Task SendMailAsync(string email)
 {
    var myMessage = new SendGridMessage();

    myMessage.From = new MailAddress("Rick@Contoso.com");
    myMessage.AddTo(email);
    myMessage.Subject = "Using QueueBackgroundWorkItem";

    //Add the HTML and Text bodies
    myMessage.Html = "<p>Check out my new blog at "
          + "<a href="http://blogs.msdn.com/b/webdev/">"
          + "http://blogs.msdn.com/b/webdev/</a></p>";
    myMessage.Text = "Check out my new blog at http://blogs.msdn.com/b/webdev/";

    using (var attachmentFS = new FileStream(GH.FilePath, FileMode.Open))
    {
       myMessage.AddAttachment(attachmentFS, "My Cool File.jpg");
    }

    var credentials = new NetworkCredential(
       ConfigurationManager.AppSettings["mailAccount"],
       ConfigurationManager.AppSettings["mailPassword"]
       );

    // Create a Web transport for sending email.
    var transportWeb = new Web(credentials);

    if (transportWeb != null)
       await transportWeb.DeliverAsync(myMessage);
 }

I set the account and password on the Configure tab in the Azure portal to keep my credentials secure.

Using the cancellation token

You can drop the following code in a new MVC app to test the cancellation token:

public class HomeController : Controller
{
   static int logCount = 0;
   public ActionResult Index()
   {
      return View();
   }

   public ActionResult About()
   {
      ViewBag.Message = "Your application description page.";
      HostingEnvironment.QueueBackgroundWorkItem(ct => workItemAction1(ct, "About"));
      return View();
   }

   public ActionResult Contact()
   {
      ViewBag.Message = "Your contact page.";
      string info = "Headers: " + HttpContext.Request.Headers.AllKeys.ToString()
              + "  URI: " + HttpContext.Request.Url.AbsoluteUri.ToString()
              + " User: " + HttpContext.User.ToString();

      HostingEnvironment.QueueBackgroundWorkItem(ct => workItem1Async(ct, "Contact"));
      return View();
   }

   private void workItemAction1(CancellationToken ct, string msg)
   {
      logCount++;
      int currentLogCount = logCount;

      ct = addLog(ct, currentLogCount, msg);
   }

   private async Task<CancellationToken> workItem1Async(CancellationToken ct, string msg)
   {
      logCount++;
      int currentLogCount = logCount;

      await addLogAsync(ct, currentLogCount, msg);
      return ct;
   }

   private CancellationToken addLog(CancellationToken ct, int currentLogCount, string msg)
   {
      Trace.TraceInformation(msg);

      for (int i = 0; i < 5; i++)
      {
         if (ct.IsCancellationRequested)
         {
            Trace.TraceWarning(string.Format("{0} - signaled cancellation", 
               DateTime.Now.ToLongTimeString()));
            break;
         }
         Trace.TraceInformation(string.Format("{0} - logcount:{1}", 
            DateTime.Now.ToLongTimeString(), currentLogCount));
         Thread.Sleep(6000);
      }
      return ct;
   }

   private async Task<CancellationToken> addLogAsync(
      CancellationToken ct, int currentLogCount, string msg)
   {

      try
      {
         for (int i = 0; i < 5; i++)
         {
            if (ct.IsCancellationRequested)
            {
               Trace.TraceWarning(string.Format("{0} - signaled cancellation : msg {1}",
                  DateTime.Now.ToLongTimeString(), msg));
               break;
            }
            Trace.TraceInformation(string.Format("{0} - msg:{1} - logcount:{2}",
               DateTime.Now.Second.ToString(), msg, currentLogCount));

            // "Simulate" this operation took a long time, but was able to run without
            // blocking the calling thread (i.e., it's doing I/O operations which are async)
            // We use Task.Delay rather than Thread.Sleep, because Task.Delay returns
            // the thread immediately back to the thread-pool, whereas Thread.Sleep blocks it.
            // Task.Delay is essentially the asynchronous version of Thread.Sleep:
            await Task.Delay(2000, ct);
         }
      }
      catch (TaskCanceledException tce)
      {
         Trace.TraceError("Caught TaskCanceledException - signaled cancellation " + tce.Message);
      }
      return ct;
   }
}

Hit F5 to debug the app, then click on the About or Contact link. Right click on the IIS Express icon in the task notification area and select Exit.

The visual studio output window shows the task is canceled.

QueueBackgroundWorkItem limitations

The QBWI API cannot be called outside of an ASP.NET-managed AppDomain.
The AppDomain shutdown can only be delayed 90 seconds (It’s actually the minimum of HttpRuntimeSection.ShutdownTimeout and processModel shutdownTimeLimit). If you have so many items queued that they can’t be completed in 90 seconds, the ASP.NET runtime will unload the AppDomain without waiting for the work items to finish.
The caller’s ExecutionContext is not flowed to the work item. For example, the code that you run in the background thread doesn’t have access to commonly used context properties. If you need HttpContext information, copy the values you care about to a state object or inside a closure and pass it in to the background worker. Don’t pass the HttpContext instance itself, as it’s not a thread-safe object and even simple property getters (like HttpContext.Request.Url) might throw.
Scheduled work items are not guaranteed to ever execute, once the app pool starts to shut down, QueueBackgroundWorkItem calls will not be honored.
The provided CancellationToken will be signaled when the application is shutting down. The work item should make every effort to honor this token. If a work item does not honor this token and continues executing, the ASP.NET runtime will unload the AppDomain without waiting for the work item to finish.
We don’t guarantee that background work items will ever get invoked or will run to completion. For instance, if we believe a background work item is misbehaving, we’ll kill it. And if the w3wp.exe process crashes, all background work items are obviously dead. If you need reliability, you should use Azure’s built-in scheduling functions.