December 14th, 2020

How Azure.com uses Serverless Functions for Consumption-based utilization and reduced always-on electric footprint

Aaron Hogue
Principal Software Engineering Manager

In June 2020, we shared how Azure.com achieves a global scale and how Sustainable Software Engineering principles were at the center. In this post we discuss how serverless architecture helped us create more sustainable apps and saved us 10x in azure spend for our data pipeline middleware. By shifting our workloads from always-on to on-demand (i.e. sustainable engineering), our carbon efficiency has significantly improved.

Always on to Consumption-Based

Migration to Serverless

In November 2020, Azure.com successfully completed the migration of dozens of data pipelines from dedicated always-on App Service WebJobs to Serverless Functions using consumption-based utilization. Our primary motivators were reduction in COGS savings, sustainable engineering, and design simplicity. Each datacenter used to host its own data pipeline for handling external data integrations. With the new strategy, we only run our data pipelines in a single geography, but use a shared service bus for business continuity. The architecture is depicted below and summarizes the before and after workflows. Azure.com data pipelines are data ingestion workflows that take external data stores and extract-transform-load (ETL) them to local regional data stores for high-availability. An example workflow for us would be the Azure.com blog. We extract data from an external data store and transform it to local data stores (e.g. cache-aside pattern) such as redis, blob storage, and search used in our page rendering layer.

WebJobs Design Architecture – Dedicated Compute

 

Azure.com regional architecture: App Service hosts regional instances and uses regional queue storage for event messaging
Azure.com regional architecture: App Service hosts regional instances and uses regional queue storage for event messaging

Serverless Design Architecture – Elastic Compute

Azure.com data pipeline architecture: Function app integration with service bus and cache-aside storage.
Azure.com data pipeline architecture: Function app integration with service bus and cache-aside storage.

Another optimization we made architecturally was adopting a push data model to reduce unnecessary API calls when no data had changed in the underlying source. Where possible, this tradeoff required tighter-coupling with external systems (using a service bus pub/sub pattern), but the payoff was increased savings as we only use compute, storage, and networking when necessary. Previously we used timer cron jobs to ping API endpoints even when data had not changed and in some cases wastefully refreshing the entire data targets.

Once we adopted function apps per data pipeline, we deprecated our legacy webjobs footprint, reducing our COGS spend 10x. This architectural refactoring reduced not only our Azure Spend but also our carbon consumption by 10x as we no longer spent wasteful cycles doing no-ops or refreshing data with the same dataset. This approach is consistent with sustainability principle of energy proportionality – specifically “maximizing utilization levels” by using elastic consumption-based compute. With push-data models we were able to make a more sustainable architecture and take a simpler design approach.

Scalable Challenges – Sustainable Wins

There were several data pipeline scalability challenges we encountered during this migration.  The first challenge that we ran into was message drops as our service bus endpoint was being throttled. Throttling only triggered under high messaging load. Our solution was to leverage service bus autoscale for elastic messaging units. The default messaging units enable a single worker, while scaling out during peak demands supported multiple service bus workers. The second challenge we overcame to further reduce network utilization was employing batched publishers and subscribers rather than sending or receiving one service bus message at a time per function. We can publish or accept a bulk number of messages. This significantly reduced the chattiness of the function / service bus interface and improved the data throughput.

Your Journey

Tell us in the comments below how you are applying sustainable principles into your application architecture. Kudos to the Azure.com engineering team who helped make this progress towards sustainability possible!

Future Roadmap

Azure.com continues to drive towards more sustainable architecture. Our next step is leveraging .NET 5.0 for our content rendering pipeline further driving towards increased request per second (RPS) per dedicated App Service node and reducing our compute footprint.

 

 

 

Author

Aaron Hogue
Principal Software Engineering Manager

Aaron is a Principal Software Engineering Manager for Azure.com marketing site and helps drive sustainable engineering practices within the marketing platform. Before Microsoft he was at an EdTech startup, consulted for both private businesses and publicly traded fortune 100 companies as well as volunteering for non-profits. When not typing on his glowing mechanical keyboard you will likely find him relaxing at the oceanside, riding a dirt bike, or kayaking in SWFL.

0 comments

Discussion are closed.