![\"\"](\"https:\/\/devblogs.microsoft.com\/java\/wp-content\/uploads\/sites\/51\/2025\/02\/email.webp\")
<\/a><\/p>\nWhy Choose Queue-Based Load Leveling?<\/h2>\n
In today’s fast-paced software environment, dynamic workloads are common. The\u00a0Queue-Based Load Leveling pattern<\/strong>\u00a0decouples the production and consumption of tasks by introducing a queue between them. This allows producers and consumers to work independently, preventing system overload during traffic spikes.<\/p>\n For small, monolithic applications, in-memory queues might suffice. However, as applications scale or adopt distributed architectures, these simple queues can become bottlenecks. In a distributed setting, relying on in-memory queues means the entire message load is handled by a single node’s memory, which can quickly become a congestion point under heavy traffic. This setup lacks scalability, fault tolerance, and load balancing, making it difficult to efficiently manage surges in workload without increased latency or potential message loss.<\/p>\n This is where\u00a0Azure Service Bus<\/strong>\u00a0comes into play. As a robust, cloud-native messaging solution, it provides the scalability, durability, and resilience that modern applications demand.<\/p>\n We’ll demonstrate the pattern with a sample application that implements the Queue-Based Load Leveling pattern in Java. Here’s what it entails:<\/p>\n Email Simulation:<\/strong>\nMultiple producers generate email tasks\u2014each with a unique ID\u2014and enqueue them into a shared\u00a0 Real-Time Status Dashboard:<\/strong>\nA Swing-based UI updates every 500 milliseconds, displaying live metrics such as current queue length, total emails produced, processed counts, and the number of active threads.<\/p>\n<\/li>\n LLM-Generated Performance Report:<\/strong>\nAt the end of the simulation, the application gathers performance metrics and crafts a prompt for LangChain4j. Using Ollama with the phi4 model, it then streams a natural language report summarizing overall performance.<\/p>\n<\/li>\n Modern Concurrency with Java 21:<\/strong>\nLeveraging Java 21\u2019s virtual threads, the application handles concurrency efficiently, reducing traditional overhead and making the code both cleaner and faster.<\/p>\n<\/li>\n<\/ul>\n Before diving in, ensure you have:<\/p>\n Java 21:<\/strong>\nLeverage the latest Java features, including virtual threads.<\/p>\n<\/li>\n Maven:<\/strong>\nManage dependencies and build the project seamlessly.<\/p>\n<\/li>\n Ollama:<\/strong>\nInstall\u00a0Ollama<\/a>\u00a0and set up the phi4 model:<\/p>\n<\/li>\n<\/ul>\n Ensure the LLM server is accessible at\u00a0 Clone the Repository:<\/strong><\/p>\n Build the Project:<\/strong><\/p>\n This compiles your source code and produces an executable JAR in the\u00a0 Start the LLM Service:<\/strong>\nEnsure Ollama is running:<\/p>\n<\/li>\n<\/ul>\n Launch the Application:<\/strong>\nStart the application with:<\/p>\n<\/li>\n<\/ul>\n A simple UI will appear with live metrics, and after a brief simulation, an LLM-generated performance report will be displayed in your console.<\/p>\n NOTE:<\/strong>\u00a0For a more detailed look into the architecture, LangChain4j<\/a> components, and workflow of the email processor app, check out the\u00a0demo.md<\/a>\u00a0file.<\/p>\n<\/blockquote>\n While in-memory queues are suitable for prototyping, enterprise-grade applications require a messaging backbone that can handle heavy loads.\u00a0Azure Service Bus<\/strong>\u00a0offers:<\/p>\n Decoupling:<\/strong>\nOffload message processing to keep your main application responsive.<\/p>\n<\/li>\n Scalability:<\/strong>\nAutomatically adjust to workload changes without overprovisioning.<\/p>\n<\/li>\n Resilience:<\/strong>\nFeatures like message persistence, automatic retries, and dead-letter queues ensure your system can handle failures gracefully.<\/p>\n<\/li>\n<\/ul>\n Azure Service Bus allows you to focus on your core business logic instead of managing low-level messaging operations. Instead of processing emails synchronously and risking performance bottlenecks during high-traffic periods, this design offloads email handling to a dedicated service. Deployed via\u00a0Azure Container Apps<\/strong>, the service continuously monitors the queue and processes messages asynchronously, ensuring reliable and scalable email management under heavy load.<\/p>\n Next, let\u2019s explore how the Contoso Fiber MWA reference application uses Azure Service Bus and Azure Container Apps to autoscale the email processor based on the number of messages in the queue.<\/p>\n The\u00a0 To simulate the autoscaling, follow the steps below:<\/p>\n Navigate to Azure App Configuration and change the\u00a0 Restart the Web App in App Service.<\/p>\n Send an email following the steps\u00a0in the Strangler Fig Pattern section of the reference application – https:\/\/github.com\/Azure\/modern-web-app-pattern-java\/blob\/main\/docs\/SranglerFig.md<\/a>.<\/p>\n<\/li>\n Navigate to the Azure Service Bus in the Azure portal. You will see a spike in incoming messages.<\/p>\n Navigate to the Container App in the Azure portal and click on the\u00a0 Integrating Azure Service Bus with queue-based load leveling empowers your Java applications with unparalleled scalability and resilience. Offloading critical operations to containerized services like Azure Container Apps keeps your system performing at its best, even under heavy loads. Embracing modern patterns like queue-based load leveling is key to building scalable, resilient applications. Integrating tools like Azure Service Bus into your Java projects can transform the way your systems handle load and scale – get started now with the\u00a0Modern Web App Pattern for Java<\/a><\/p>\n In this blog, we\u2019re exploring the Queue-Based Load Leveling pattern\u2014a powerful method for smoothing out workload spikes Why Choose Queue-Based Load Leveling? In today’s fast-paced software environment, dynamic workloads are common. The\u00a0Queue-Based Load Leveling pattern\u00a0decouples the production and consumption of tasks by introducing a queue between them. This allows producers and consumers to work independently, […]<\/p>\n","protected":false},"author":29534,"featured_media":231916,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[816,14,1,8,15],"tags":[],"class_list":["post-231915","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-container-apps","category-cloud","category-java","category-open-source","category-vscode"],"acf":[],"blog_post_summary":" In this blog, we\u2019re exploring the Queue-Based Load Leveling pattern\u2014a powerful method for smoothing out workload spikes Why Choose Queue-Based Load Leveling? In today’s fast-paced software environment, dynamic workloads are common. The\u00a0Queue-Based Load Leveling pattern\u00a0decouples the production and consumption of tasks by introducing a queue between them. This allows producers and consumers to work independently, […]<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/posts\/231915","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/users\/29534"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/comments?post=231915"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/posts\/231915\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/media\/231916"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/media?parent=231915"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/categories?post=231915"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/java\/wp-json\/wp\/v2\/tags?post=231915"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nOverview of the Sample Application<\/h2>\n
\n
BlockingQueue<\/code>. Consumers process these tasks asynchronously. A \u201cpoison pill\u201d mechanism ensures that consumer threads terminate gracefully when their work is done.<\/p>\n<\/li>\n
\nGetting Started<\/h2>\n
Prerequisites<\/h3>\n
\n
ollama run phi4<\/code><\/pre>\n\n
http:\/\/localhost:11434<\/code>.<\/p>\n<\/li>\n<\/ul>\nInstallation Steps<\/h3>\n
\n
git clone<\/span> https:\/\/github.com\/roryp\/queue-load-leveling\r\ncd<\/span> queue-load-leveling<\/code><\/pre>\n<\/li>\nmvn clean package<\/code><\/pre>\ntarget<\/code>\u00a0directory.<\/p>\n<\/li>\n<\/ol>\nRunning the Application<\/h3>\n
\n
ollama run phi4<\/code><\/pre>\n\n
java -jar target\/QueueLoadLevelingWithVirtualThreads.jar<\/code><\/pre>\n\n
<\/a><\/p>\n<\/li>\n<\/ul>\n\n
\nAdvancing to Azure Service Bus<\/h2>\n
\n
MWA reference application Autoscaling Email Processor<\/h2>\n
email-processor<\/code>\u00a0container app is configured to autoscale based on the number of messages in the Azure Service Bus. The\u00a0email-processor<\/code>\u00a0container app scales out when the number of messages in the Service Bus exceeds a certain threshold.<\/p>\n<\/a><\/p>\n\n
CONTOSO_SUPPORT_GUIDE_REQUEST_SERVICE<\/code>\u00a0value to\u00a0demo-load<\/code>.<\/p>\n<\/a><\/p>\n<\/li>\n<\/a><\/p>\n<\/li>\n<\/a><\/p>\n<\/li>\nRevisions and replicas<\/code>\u00a0link under\u00a0Application<\/code>\u00a0in the left navigation. Finally, click on the\u00a0Replicas<\/code>\u00a0tab. You will see that the number of replicas has increased.<\/p>\n<\/a><\/p>\n<\/li>\n<\/ol>\nConclusion<\/h2>\n
\n","protected":false},"excerpt":{"rendered":"