Ordering Messages in Azure Service Bus
Application Development Manager Herald Gjura discusses how to enforce message ordering in an Azure Service Bus queue or topic while retaining a multi-publisher / multi-subscriber architecture for high message volume throughput.
The Scenario
One of the most asked questions, every time I speak about Azure Service Bus, is: “How do you force ordering of messages in an Azure Service Bus queue or topic.” In this post, I will answer this and provide a solution for processing Azure Service Bus messages in a predefined order.
The default mode, also its super-power, for Azure Service Bus is multi-publisher/multi-subscriber, or fan-in/fan-out, where we pump messages in at a furious pace from multiple sources. We then process them using multiple threads or multiple processes. This is a great way to build decoupled solutions with a clear “separation of concerns” architecture.
As good as a scenario this is, it does come with some drawbacks. What do we do if we have a need for messages to be processed in a certain order? For example, you are processing an e-commerce order, and you have all the order items coming in as messages in a service bus queue at the warehouse. Or, if you are processing each step that a package follows in a predefined fulfillment workflow!
While Azure Service Bus allows for a FIFO approach (First-In-First-Out), we cannot guarantee that messages are entered in the order we want them to be processed by a Subscriber process. To do that, we would need to sacrifice the multi-threaded / multi-process (Fan-In) way of entering messages and instead adopt a single-Publisher approach.
Similarly, on the message processing side we cannot guarantee a thread-safe approach to processing messages without adopting a single-Subscriber approach, thereby sacrificing the scalability benefits of the Fan-Out approach.
Each of these options, will diminish the rich features of the Azure Service Bus, reducing the power of a true Service Bus architecture.
The Solution
Azure Service Bus does not have a native solution or feature to cover this scenario. But don’t let that stop you! There are other features that we can put together to accomplish this.
Sessions
We can force the messages that are tagged with the same SessionId to be grouped together and be delivered in a FIFO manner. Note that by FIFO here it is meant the order in which the messages have arrived at the Azure Service Bus, and not necessarily the internal order defined by you that you want the Subscriber to follow when processing the messages.
How Sessions work in Azure Service Bus is best illustrated by the following diagram:
An example of session grouping would be: if you are processing Order Line Items, then you should use OrderId to set the SessionId and force all Order Line Items of an Order to be processed together. Alternatively, if you are processing Packages in a fulfillment warehouse that go through steps in a workflow, then you can use PackageId, or WorkflowId (if multiple packages or items are involved).
Be mindful when you set the Session and SessionId. If you create a very large session, that will force Azure Service Bus to send most messages to one Subscriber, reducing the multi-threading benefits. If you set sessions too granular, then it loses its intended benefit, and you are simply adding unnecessary overhead.
SessionState
Each Session has a SessionState object. We can use this to store the metadata for our processing progress. You can look at the code to see how this feature is used. To illustrate this, in the accompanying code samples, I have created a custom SessionStateManager object to store all the info we need about the object. More on this later.
Message Deferral
Deferring a message is an interesting feature in Azure Service Bus. Consider this scenario: we have an Order we need to process, with Order Line Items that are ordered from 1 to 10, and need to process following that sequence (Line Item 2 cannot be processed before Line Item 1, etc.).
We fire up a subscriber, which hooks into a session and picks up the first message in the queue. It happens to be Order Line Item #3. We were expecting Order Line Item #2.
We cannot process this message. But we cannot also put it back in the queue, as it will show up immediately at the front of the queue. We cannot place it in the Deadletter subqueue, as there is nothing wrong with the message. We defer it, using DeferAsync(). This way we put the message back in the main queue, but it makes it invisible to the Subscriber.
A deferred message cannot be picked by a Subscriber through its normal operations. To retrieve a deferred message, we need to know its SequenceId. Furthermore, we are responsible for managing and maintaining this SequenceId, in order to retrieve the deferred message. If we lose the SequenceId, then the message is lost forever, and upon expiration of the Time-to-Live (TTL) it will end up in the Deadletter queue as an expired message.
This is where the SessionState above comes in handy. In it, we can store the SequenceIds for all the deferred messages we want to retrieve.
Session Closure
Whenever we reach the end of the session, it is important that we close the session object. But in order to do so, we need to know that the last item in the sequence has been processed. We can do that by setting a flag in the last message in the sequence. Once we have processed the final message, we close the session. See it at work in the code.
The Code
The accompanying code sample is available here, on GitHub, and illustrates all the above concepts.
This sample code is comprised of the following files:
- SessionStateManager.cs: Contains the custom class that manages the Session State.
- Publisher.cs: Creates a series of dynamic objects, sets their order/sequence, and pumps them in a topic in Azure Service Bus. It does this in a multi-threaded way, simulating a multi-Publisher scenario, and the order that messages go into the Azure Service Bus is scrambled and uncertain.
- Subscriber.cs: Creates multiple listeners to retrieve messages, manages the Session State, and handles the message processing in the desired sequence.
- Program.cs: The entry point of this console code sample. You can customize it further to create more complex scenarios.
Hopefully the solution and the code are self-explanatory.
Conclusion
This was a post to illustrate how we can force an ordering of messages to be processes in Azure Service Bus. It is by no means a production-ready code or solution. You will need to add additional code and make it cover more use-cases, as it best fits your solutions.
The intent of this article was to illustrate some of the existing and exciting features of Azure Service Bus, and hopefully this post has done just that.
Enjoy!
Light
Dark
0 comments