Special thanks to Ramya Achutha Rao for collaborating on this blog post.
With the recent release of the new client libraries for Azure Event Grid, the Azure SDK Team has released new client libraries for three key messaging services on Azure – the other two being Azure Event Hubs and Azure Service Bus. The libraries all follow the same design guidelines and are available in .NET, Java, JavaScript/TypeScript, and Python. This blog post will highlight some of the key improvements in the new libraries.
You can find links to all the latest libraries at https://aka.ms/azsdk.
Why new client libraries?
The new client libraries for messaging services are part of a larger set of new client libraries that we have been releasing for different Azure services since November 2019. We have shared the motivation behind the creation of these new libraries in the following blog posts: Previewing Azure SDKs following new Azure SDK API Standards, and State of the Azure SDK 2021.
Messaging services on Azure
Although Azure Event Hubs, Azure Service Bus, and Azure Event Grid share some similarities, they are designed for different scenarios. For a detailed comparison of the services, we recommend checking out these articles in the documentation for Azure services:
- Choose between Azure messaging services – Event Grid, Event Hubs, and Service Bus
- Events, Data Points, and Messages – Choosing the right Azure messaging service for your data
Azure Event Hubs SDK Highlights
Simpler clients, consistent experience
Clients are usually the entry point into a library. This means before users can invoke a method, they have to create an instance of a client first. We have redesigned the client hierarchy of the new libraries to be simpler and more approachable for developers. The new libraries use the same client for all publishing scenarios, with a set of options controlling specific behaviors rather than having different clients for each scenario. This design, along with having similar names, signatures, and location for the methods provide a consistent experience for publishing. Please see our Migration Guides for a more detailed comparison of the client hierarchy in the latest library and that of the previous library.
Easily create optimally sized batches
Publishing in batches gives better efficiency and throughput. While this feature was available before, it is more sophisticated in the new libraries. Users no longer have to wait for the send method to fail to realize that a batch has exceeded its size limit. Instead, when they add events into a batch, they are notified if the addition of a particular event causes the batch to exceed its size limit. This eliminates the need for trial-and-error when trying to send batches optimally.
Easily customize durable store for checkpoints
We redesigned the way checkpoint store support is implemented in the new libraries to make it easier for users to swap out Azure Blob Storage with a durable store of their choice for storing checkpoints.
Language-specific Highlights
The new Java library provides Reactor support, just like all the other new Java libraries for Azure services.
For receiving events, the new Java and .NET libraries provide two clients, the consumer client and the processor client for different application needs. The consumer client allows users to receive from a single partition and gives them a higher degree of control but at the cost of additional complexity around partition management. The processor client provides a more approachable onboarding experience for new users by eliminating the need to manage partitions. It also offers a clear step-up path to production where load-balancing across partitions would become necessary. Learn more about the processor client in the following blog posts:
- Process data in real-time with Java and Azure Event Hubs
- How to choose the right Azure Event Hubs .NET client library type for your application?
- Building a custom Event Hubs Event Processor with .NET
A commonly reported issue for the old JavaScript library was the inability to control the pace at which events are received, which could result in flooding of downstream dependencies and confusions about which events have been consumed and in what order. The new JavaScript library fixed this issue so that users get a new event only after they finish processing the current one.
The new JavaScript and Python libraries consolidated multiple ways of receiving events into a single method that works for receiving from one partition as well as all partitions, and with or without checkpointing. These libraries also allow configuration of retry policy used by operations on the clients, which was a feature not available before.
Language | New Package | Samples for new packages | Old Package(s) that are being replaced |
---|---|---|---|
Java | azure-messaging-eventhubs | Samples for azure-messaging-eventhubs | azure-eventhubs |
.NET | Azure.Messaging.EventHubs | Samples for Azure.Messaging.EventHubs | Microsoft.Azure.EventHubs, WindowsAzure.ServiceBus* |
Python | azure-eventhub | Samples for azure-eventhub | Versions prior to version 5 of the same package |
JavaScript/ TypeScript | @azure/event-hubs | Samples for @azure/event-hubs | Versions prior to version 5 of the same package |
*Event Hubs and Service Bus once shared the same library
Please see Migration Guides for more detailed information on these new libraries and how to migrate from the old ones.
Azure Service Bus SDK Highlights
Simpler clients, consistent experience
Since clients are usually the entry point into a library, we simplified the client hierarchy of the new Service Bus libraries to be simpler and more approachable. Clients are the entry point because before users can invoke a method, they typically have to create an instance of a client first. The old libraries had separate clients for sending and receiving messages from queues, topics, and subscriptions. In addition, they had an advanced sender, an advanced receiver, and a separate client for managing Service Bus resources. The new libraries, however, have a single client from which users can create senders and receivers for queues, topics, and subscriptions. The separate client for managing resources is kept for the purpose of achieving a clear separation of responsibilities. Please see our Migration Guides for a more detailed comparison of the client hierarchy in the latest library and that of the previous library.
Receive from deadletter queue easily
The new libraries make it easier to receive from dead letter queues or transfer dead letter queues, which are essentially sub queues in a queue or subscription. Users would create a receiver like they normally do and then specify in the option parameter to use the sub queue.
Access to underlying AMQP message
A Service Bus message is converted to an AMQP message before it is sent to the service. However, the old libraries did not allow users to read all AMQP message properties, only those specific to a Service Bus message could be read. The new libraries removed this restriction, allowing users to read the underlying AMQP message with all its details as per the AMQP specification. This is helpful for users who have been using a different library to send messages to Azure Service Bus and want to take advantage of the full AMQP message format.
Create optimally sized batches easily
Batching improvement in the new Service Bus libraries is very similar to that of the new Event Hubs libraries. APIs are provided to help users create optimally-sized message batches for better efficiency and throughput. Batching is more sophisticated in the new libraries because users no longer have to wait for the send method to fail to realize that a batch has exceeded its size limit. Instead, they are notified if the addition of a particular message causes the batch to exceed its size limit when adding messages into the batch. This means users don’t need to guess when trying to send batches optimally.
Language-specific Highlights
Users can use receivers to receive messages in the new .NET and Java libraries just like in the old libraries, but they also have the option of choosing processors in the new libraries. Processors give users less control over receiving as compared to receivers but provide a simple way to get started. They can be used to process messages concurrently with the assurance of failure resilience.
The new JavaScript library provides the ability to manage Service Bus resources, i.e. create, get, delete, update, and list queues, topics, subscriptions, and rules. This was a feature gap in the old JavaScript library because all the other languages provided that.
The new Python library allows users to connect to Azure Service Bus via a proxy, which was a feature available in the other languages but not in Python previously.
Since all new Java libraries for Azure services have Reactor support, the new Service Bus library does as well.
Language | Package | Samples for new packages | Old Package(s) |
---|---|---|---|
Java | azure-messaging-servicebus | Samples for azure-messaging-servicebus | azure-servicebus |
.NET | Azure.Messaging.ServiceBus | Samples for Azure.Messaging.ServiceBus | Microsoft.Azure.ServiceBus, WindowsAzure.ServiceBus* |
Python | azure-servicebus | Samples for azure-servicebus | Versions prior to version 7 of the same package |
JavaScript/ TypeScript | @azure/service-bus | Samples for @azure/service-bus | Versions prior to version 7 of the same package |
*Event Hubs and Service Bus once shared the same library
Please see Migration Guides for more detailed information on these new libraries and how to migrate from the old ones.
Azure Event Grid SDK Highlights
Support for Event Grid Event, Cloud Event, and Custom Event
The new libraries support Event Grid Event, Cloud Events, and Custom Event whereas the old libraries supported only the first type that is native to the Event Grid service itself. Cloud Events is a universal specification of events independent of providers and is maintained by the Cloud Native Computing Foundation (CNCF). Custom Event is user-defined; it is useful for situations where one needs to deal with events that do not match the other two types. Because Cloud Events is the open standard, more investments will be made on it in the future.
Trace how Cloud Events move through different components of an application
Since Cloud Events allows for metadata injection, a unique marker can be added to the event as it moves through different components of an application. This is done automatically if users enable distributed tracing, which is provided by the new libraries.
Support for authentication using Shared Access Signature
The new libraries support for authentication using Shared Access Signature (SAS) tokens, which is useful for scenarios where users don’t need to be given full access, but only time-based access. The new libraries also provide APIs not found in previous versions that help users generate the SAS token.
Use same credentials to publish to topics and domains
In the new libraries, users don’t need to worry about changing credentials when switching from publishing to a topic to publishing to a domain as they would have to in the old libraries.
Language-specific Highlights
The new .NET library comes with a bridge library that provides integration with CNCF’s .NET CloudNative.CloudEvents library for those interested in using both.
The new TypeScript library makes it much easier to consume events sent by Azure services as it has the ability to understand the types and properties of these system-generated events. In the old libraries, users would have to figure out this information by using documentation.
Finally, there is support for Reactor in the new Java library as mentioned in the other sections.
Language | Package | Samples for new packages | Old Package(s) |
---|---|---|---|
Java | azure-messaging-eventgrid | Samples for azure-messaging-eventgrid | azure-eventgrid |
.NET | Azure.Messaging.EventGrid | Samples for Azure.Messaging.EventGrid | Microsoft.Azure.EventGrid |
Python | azure-eventgrid | Samples for azure-eventgrid | Versions prior to version 4 of the same package |
JavaScript/ TypeScript | @azure/event-grid | Samples for @azure/eventgrid | Versions prior to version 4 of the same package |
Please see Migration Guides for more detailed information on these new libraries and how to migrate from the old ones.
Migration Guides
Please see our migration guides for guidance on how to migrate to the latest libraries. The guides also detail what features and improvements are made in the new libraries.
Event Hubs
- Java Migration Guide
- Python Migration Guide
- JavaScript/TypeScript Migration Guide
- .NET Migration Guide
Service Bus
- Java Migration Guide
- Python Migration Guide
- JavaScript/TypeScript Migration Guide
- .NET Migration Guide
Event Grid
- Java Migration Guide
- Python Migration Guide
- JavaScript/TypeScript Migration Guide
- .NET Migration Guide
Azure SDK Blog Contributions
Thank you for reading this Azure SDK blog post! We hope that you learned something new and welcome you to share this post. We are open to Azure SDK blog contributions. Please contact us at azsdkblog@microsoft.com with your topic and we’ll get you set up as a guest blogger.
Azure SDK Links
- Azure SDK Website: aka.ms/azsdk
- Azure SDK Intro (3 minute video): aka.ms/azsdk/intro
- Azure SDK Intro Deck (PowerPoint deck): aka.ms/azsdk/intro/deck
- Azure SDK Releases: aka.ms/azsdk/releases
- Azure SDK Blog: aka.ms/azsdk/blog
- Azure SDK Twitter: twitter.com/AzureSDK
- Azure SDK Design Guidelines: aka.ms/azsdk/guide
- Azure SDKs & Tools: azure.microsoft.com/downloads
- Azure SDK Central Repository: github.com/azure/azure-sdk
- Azure SDK for .NET: github.com/azure/azure-sdk-for-net
- Azure SDK for Java: github.com/azure/azure-sdk-for-java
- Azure SDK for Python: github.com/azure/azure-sdk-for-python
- Azure SDK for JavaScript/TypeScript: github.com/azure/azure-sdk-for-js
- Azure SDK for Android: github.com/Azure/azure-sdk-for-android
- Azure SDK for iOS: github.com/Azure/azure-sdk-for-ios
- Azure SDK for Go: github.com/Azure/azure-sdk-for-go
- Azure SDK for C: github.com/Azure/azure-sdk-for-c
- Azure SDK for C++: github.com/Azure/azure-sdk-for-cpp
These are nice, but regarding the service bus library, I think its still missing plugin support. That is a major omission from my perspective and I would suspect others as well. I know I can’t migrate until that is resolved.
now i see https://github.com/Azure/azure-sdk-for-net/issues/11986
left a thumbs up for my vote
Thanks for Christopher for the feedback! Our engineering team is looking at a few ways to provide support for the plugin scenario. Here’s a GitHub issue on the design discussion we have: https://github.com/Azure/azure-sdk-for-net/issues/12943