Creating a Single Bot Service with Azure to Support Multiple Bot Applications
Cover image used with permission from moed.ai
This post describes a method to create a single bot service that runs on Azure and serves multiple bot applications. One of the benefits of this architecture is the ability to manage it as a single service instead of creating a separate bot service for each bot application. Instead, you can scale multiple services up or out, or manage payments and more at the same time. In this case, our bot service is configuration-based, so it is easy to write its engine and then apply it to all the different bot applications. You can also apply this approach to create a single service that hosts multiple bots and routes incoming requests to the relevant bot’s logic.
Background
moed.ai offers a platform for scheduling services. Their customers can configure their services, resources and calendars using a designated dashboard. Resources can be objects such as cars or meeting rooms, as well as people like test drivers or sales representatives in an automobile agency. Each of these resources has a calendar defining its working hours that the platform manages and uses to schedule meetings with clients of moed.ai’s customers.
moed.ai’s scheduling platform is configuration-based, as there is no customer-specific code implementation. The same engine serves all of moed.ai’s customers.
In addition to the current web interface that allows moed.ai’s customers’ clients to schedule meetings, moed.ai wanted to create a Scheduling Assistance Bot for their customers to embed on their websites or Facebook pages. Since the platform is configuration- or rule-based, they wanted to create a single bot that will be used by all of their customers.
The Problem
A bot entity maps to a registration entry in the Bot Framework Developer portal and can be published independently on any of the supported channels. All of the existing Microsoft Bot Framework samples today demonstrate how to create a bot by implementing a bot service deployed on Azure. They also show how to implement a bot entity in the Bot Framework Developer portal, pointing to the bot service deployed on Azure.
To create one bot entity in the Bot Framework Developer portal and publish it on multiple websites or Facebook pages, we had to come up with a way to pass the customer’s ID along with each request that is sent to the bot service through the Bot Framework, so that the bot service will know who the relevant customer is, and use its configuration to process the request.
The problem was that there is currently no way to pass the customer’s ID along with the requests to the bot service.
The Solution
As part of registering a new bot in the Bot Framework Developer portal, we also must provide a Messaging Endpoint that is used by the Bot Framework to send messages to the bot service:
Our solution makes use of this Messaging Endpoint, by adding the customer’s ID to the Messaging Endpoint path.
For this approach, we need to register a new bot in the Bot Framework Developer portal for each of moed.ai’s customers, so that we will be able to provide a different Messaging Endpoint that encapsulates their ID:
Unfortunately, there is currently no API to programmatically create these bot registrations in the Bot Framework Developer portal. For the time being, a new bot will have to be created manually for each customer in the Bot Framework Developer portal.
For each of the bot entries in the Bot Framework Developer portal, we will have to maintain its application ID and password and expose the corresponding Messaging Endpoint REST API on our bot service. By having a different bot entity for each customer, the bot can be published on any of the channels that a customer requires (a Facebook page, a website, etc.).
The main advantage of using this approach is that we only maintain and manage one bot service and we can also scale all of the bots together, by scaling that single bot service up or out.
The Code
Once we set up multiple bots on the Bot Framework Developer portal, we will implement a bot service that exposes REST APIs for each of these bots’ entries:
var express = require('express');
var builder = require('botbuilder');
var port = process.env.PORT || 3978;
var app = express();
// a list of client ids, with their corresponding
// appids and passwords from the bot developer portal.
// get this from the configuration, a remote API, etc.
var customersBots = [
{ cid: 'cid1', appid: '', passwd: '' },
{ cid: 'cid2', appid: '', passwd: '' },
{ cid: 'cid3', appid: '', passwd: '' },
];
// expose a designated Messaging Endpoint for each of the customers
customersBots.forEach(cust => {
// create a connector and bot instances for
// this customer using its appId and password
var connector = new builder.ChatConnector({
appId: cust.appid,
appPassword: cust.passwd
});
var bot = new builder.UniversalBot(connector);
// bing bot dialogs for each customer bot instance
bindDialogsToBot(bot, cust.cid);
// bind connector for each customer on it's dedicated Messaging Endpoint.
// bot framework entry should use the customer id as part of the
// endpoint url to map to the right bot instance
app.post(`/api/${cust.cid}/messages`, connector.listen());
});
// this is where you implement all of your dialogs
// and add them on the bot instance
function bindDialogsToBot (bot, cid) {
bot.dialog('/', [
session => {
session.send(`Hello... I'm a bot for customer id: '${cid}'`);
}
]);
}
// start listening for incoming requests
app.listen(port, () => {
console.log(`listening on port ${port}`);
});
Explanation of Code
We are creating different bot and connector instances that capture the App ID and password for each customer, and binding it to the corresponding REST API that is used by the Bot Framework as the Messaging Endpoint.
When we create the bot instance, we call the bindDialogsToBot method, passing the bot instance and the customer ID. By doing that, we capture the customer ID in its closure making it accessible to the internal dialogs.
When a call is made to one of the REST APIs, the relevant bot instance is used, and the correct customer ID can be utilized by the dialog’s internal logic to process the request (for example, to retrieve a customer’s configuration/rules and act upon them).
Opportunities for Reuse
We could reuse this approach in similar scenarios to this one where a configuration-based platform serves multiple bots. Another possible case for reuse would be a scenario where one service hosts different bots. In that sort of situation, we will need to have different dialogs for each bot based on the client ID.
Light
Dark
10 comments
Good article Ami,
In this approach do you create a Facebook App for each customer to connect with the bot developer portal or do you manage the customer pages from a Facebook app you own ?
Thanks Eddie.
Since you have a different app id in the bot developer portal, you’ll need a different fb app to map to each of these app ids.
The above written code is using Node.js . Can the same thing be done using .NET ?
Hi Nupur,
I didn’t try myself, but I don’t see a reason why this won’t be doable using the .NET SDK.
Thanks,
Ami.
I tried to replicate that script in Azure Bot service. But I kept getting this error:
Exception while executing function: Functions.messages. mscorlib: Unable to determine function entry point. If multiple functions are exported, you must indicate the entry point, either by naming it ‘run’ or ‘index’, or by naming it explicitly via the ‘entryPoint’ metadata property.
I suspect it’s the if (useEmulator) {} function
Any help?
Not sure; will take a look. For now checkout this link, although may not be same root cause. https://stackoverflow.com/questions/43702177/unable-to-determine-function-entry-point
Question – how do you push updates to the bot instances in this design? I.E. as I continue to expand my bot functionality I want my customers to benefit from the updates. Does this design allow for pushing of updates to the bot itself for each customer?
Thanks,
Lee
In the end this is one single service that serves multiple instances. There are different instances of bot and connectors for each customer.
If you need to update your customer’s functionality, you’ll need to restart the service. You can also use continuous integration that does that automatically for you.
BTW- you can use an approach similar to what I did here: https://www.microsoft.com/reallifecode/2016/12/12/integrating-an-existing-bot-platform-with-microsofts-bot-framework/ to reload your bots on demand without restarting the service, but that’s only possible if your dialogs are created using configuration, as described in this link.
Hi Ami, I wanted to check in a see how this solution turned out for you? We’re interested in developing a bot to support over 1,000 pages and are trying to estimate the effort (in terms of hours) for taking a fully developed bot and connecting it up to the relevant pages. It would be great to hear your experience/s and connect!
Hi Adam, the proposed solution is fully scalable to support as many as bots as you need. In the end the bot is a web app that can be scaled using the app service scaling mechanism Azure provides.