Orchestrating Multiple Bots with Multilingual Support

Mor Shemesh

This code story describes how my team at Microsoft collaborated with Sage to find imaginative solutions for their bot-related challenges. Sage was setting out to develop a large collection of bots that would be offered in a variety of languages, locations, and needs. But in order to build them, they first needed good design and infrastructure.

The Challenge

Sage wanted to develop a platform where they could offer bots in multiple languages and through numerous channels such as Facebook Messenger, Slack, etc.

Prior to our engagement, this partner supported its localization and channel needs by writing multiple copies of the same bot for every language and every channel. Each of these bots needed to be registered and maintained separately.

That’s a lot of code to maintain!

However, we can enable support for multiple channels as part of the registration process by using Microsoft Bot Framework. In this way, we can support publishing a single bot that:

• Enables context switching, which means that the user can switch between different child bots, like for example Credit Card Bot and Health Checkup Bot.
• Enables development with a multilingual approach using language resources and LUIS

General Solution

In our solution, the Orchestration Bot (a.k.a. Parent Bot) manages the place in the conversation where the user can decide on general context actions like:

• Changing the language
• Showing what bots are available

The Child Bot is a bot managing its own conversation with the user. Once the user enters a conversation with the Child Bot (Credit Card Bot for example), commands like help and tell me my balance will be handled by the Child Bot. If the user wants to return to the Parent Bot he or she can say something like exit or go home. They can also jump directly to a different Child Bot with a command like go to health bot.

We made use of the following features in Microsoft Bot Framework to help with the technological challenges of creating and maintaining a multilingual, context-switching bot:

1. Microsoft Bot Framework Middleware
2. Microsoft Bot Framework Libraries
3. Microsoft Bot Framework Localization

Conversational Flow

The sections below delve into these specific parts of the solution.

Using middleware to intercept messages

Middleware is implemented in Microsoft Bot Framework as a message interceptor for messages sent to the bot. Our solution uses middleware to intercept all messages and look for special ones like:

• “go home” – exit the current bot and return to the top-level bot.
• “go feedback” – exit the current bot and enter the “feedback” bot.

This method can be used to intercept messages that should be relevant for all bots.

The following is an example of how to intercept a “go home” message:

var builder = require('botbuilder');

bot.use({
    botbuilder: function (session, next) {

        if (session.message.text === 'go home') {
            
            // In case a dialog is currently active, end it and activate dialog with id '/'
            if (session.sessionState.callstack && session.sessionState.callstack.length) {
                session.cancelDialog(0, '/');

            // activate dialog with id '/'
            } else {
                session.beginDialog('/');
            }
        }
    }
});

Middleware is used in the code to orchestrate exits from and entrances to conversations from anywhere in the application which enables the user to change his or her mind and switch between bots.

Microsoft Bot Framework Libraries

The use of libraries helps separate the contexts of the parent bot and child bots and enables different teams to maintain their own code bases.

Libraries is a feature in Microsoft Bot Framework that enables the creation of segregated dialogs and intents that can function as a subcontext for the bot. That is, a child bot orchestrated by the parent bot.

In our scenario, each child bot handles a topic or domain, credit card, history, alarm, or giving feedback. By separating these domains into individual child bots, we enable the team responsible for each one to work independently in their own code base. Each child uses a library to manage the intents for the domain.

Placing the library inside a separate directory ensures that it is self-contained. The following code is an example of how to create a library inside a dedicated file:

/bots/feedback/index.js:

var feedbackBot = (function () {

  var _intents;
  var _lib = new builder.Library('feedbackBot');
  
  _lib.dialog('hello', [
      function (session, results) {
          session.endDialog('hello from feedback bot');
      }
  ]);

  function createLibrary () {
      return _lib;
  }

  function initialize (locale) {
      intents = new builder.IntentDialog();
      intents.matches(/^(hello)/i, "feedbackBot:hello");
      return intents;
  };

  return {
    createLibrary: createLibrary,
    initialize: initialize
  };

})();

module.exports = feedbackBot;

Each bot creates and manages a Library object and exposes an intents array. The parent bot will have to perform the following steps to consume the library:

// 1. Declare the library
bot.library(childBot.createLibrary());

// 2. Initialize the library and receive intents
var botIntents = selectedBot.initialize(/*...*/);

// 3. Create a new dialog with the received intents
bot.dialog('bot_name', botIntents);

// 4. Enter the dialog when needed
session.beginDialog('bot_name');

Automatic Loading of Bots

Since we will potentially need to support an extensive collection of bots, we looked for an approach that didn’t require us to declare each bot separately. To apply this method, we require each bot to implement an interface that exposes the following methods:

• createLibrary – creates and returns the library object for declaration.
• getName – returns the name of the bot that can be presented to the user.
• welcomeMessage – returns a welcome message that will be displayed to the user when the conversation begins.
• initialize – initializes the library with any runtime parameters and returns the library’s intents.

In addition to the interface we also assume that all bot directories are found under the /bots/ directory, enabling automatic loading:

// Declare all libraries directly under '/bots' directory
var bots = [];
var botDirectories = getDirectoriesUnder('./bots');
for (var dirIdx in botDirectories) {

    var dirName = botDirectories[dirIdx];
    var childBot = require(path.join(__dirname, 'bots', dirName));
    bots.push(childBot);
    bot.library(childBot.createLibrary());
}

Creating and starting a bot

To decide which bot we want to activate we offer the user a choice from the collection of loaded bots:

var botNames = bots.map(function (bot) { return bot.getName(session); });
builder.Prompts.choice(session, 'Which bot would you like to choose?', botNames);

Note: This implementation was chosen for simplicity even though it doesn’t hide the “botness” of the bot application from the user. Another better solution would be to use LUIS to understand which child bot is the appropriate one to process the received message and direct the message to it.

Then we create and launch their chosen bot:

var selectedBot = bots.find(function (bot) { return bot.getName(session) == requestedBot; });
var locale = session.preferredLocale();
var botKey = 'locale-' + locale + '-' + requestedBot;
var localeIntents = selectedBot.initialize(locale);

bot.dialog(botKey, localeIntents);
session.send(selectedBot.welcomeMessage(session));
session.beginDialog(botKey);

We used the selected locale in the bot name to enable the same bot to be declared and launched in different languages.

Enabling Multiple Languages

To communicate with a user in a way he or she understands, we need to “listen” and “reply” in his or her language. Fortunately, Microsoft Bot Framework has a solid approach for storing and displaying texts in multiple languages.

However, one major gap that is yet to be addressed in the framework is expanding LUIS, a service for intent and entity recognition, to support multiple languages through one model. To do that with Microsoft Bot Framework using Node.js, we needed to take the following requirements into account:

1. Enable the user to change the bot language in real time.
2. Use LUIS models that are paired with the LuisRecognizer and IntentDialog objects after creation.

The following are three solutions we used to enable multilingual support.

Choosing a Language

To communicate with the user in his or her desired language, we begin by offering them a collection of supported languages. If the user already has a supported language in their session.userData, that language is automatically used. Alternatively, if the user requests to “change language” we prompt them to select a new one. This logic is implemented in the parent bot since it is relevant to all parent and child bots in the application.

To set the default language for the Microsoft Bot Framework, we use the following code:

    session.preferredLocale(session.userData["BotBuilder.Data.PreferredLocale"]);
    bot.settings.localizerSettings.defaultLocale = session.preferredLocale();

Parent Bot-Level Multilingual Support

Microsoft Bot Framework enables saving JSON format resource files under /locale/{lang}/index.json where lang is a two-digit locale.

Example of storing a value and retrieving it:

/locale/en/index.json

{
    "welcome-message": "Hello and welcome to our bot!"
}

function promptWelcomeLocale() {
    session.send('welcome-message');
}

It is also possible to override default Microsoft Bot Framework messages by adding the file /locale/{lang}/BotBuilder.json. Possible values are available on GitHub.

Child Bot-Level Multilingual Support

The same method for the root level can also be applied at the library level when adding the locale file under /{bot lib}/locale/{lang}/index.json.

When initializing the library, call the following code:

  var _lib = new builder.Library('feedbackBot');
  _lib.localePath('./bots/bot_name/locale/');

This code ensures that this directory is added to the available locale. At the time of writing this article, values in such locale files are registered at the global level and override existing values. To add new values, it is advisable to use the format {bot name}:{locale value} or any other similar format.

LUIS models

A bigger challenge on the language front is using LUIS models. It’s one thing to display text for a user in their preferred language, but it’s quite another to understand a user in multiple languages. At the time of writing this article, LUIS does not support multilingual features.

This issue means that several LUIS models have to be trained in order to add multilingual support, including one for each different language. Additionally, building LUIS intent-based dialogs in Microsoft Bot Framework requires the declaration of the model in the instantiation:

    recognizer = new builder.LuisRecognizer(LUIS_MODEL_URL);
    intents = new builder.IntentDialog({ recognizers: [recognizer] });

Since the flow of dialogs in Microsoft Bot Framework is designed as a stack, you can create dialogs and route a conversation into them in runtime. To ensure that the LUIS model is available when bots are created, each library exposes an initialization method that accepts a locale and returns a collection of intents.

Having an initialization method enables the dynamic creation of the intent using a locale chosen by the user:

    var botKey = 'locale-' + locale + '-' + requestedBot;
    var localeIntents = selectedBot.initialize(locale);
    rootBot.dialog(botKey, localeIntents);
    session.beginDialog(botKey);

Using the user’s locale and dialog id we can route him or her request to the relevant bot.

Opportunities for Reuse

You could reuse the code presented here in any project involving any bot situations similar to these:

1. A bot for managing multiple contexts or child bots (see example below)
2. A bot that needs multilingual support (with or without LUIS)
3. A bot that requires runtime creation and triggering of dialogs

It would also be beneficial to reuse the knowledge gained from this case study when implementing a bot that involves one or more of the following scenarios:

• A credit card context with actions like balance inquiry, money transfer and action history
• A music context in which the user can ask things like who wrote candle in the wind, when was the wall by Pink Floyd published and purchase Bad by Michael Jackson
• An entertainment context that enables requests like tell me a joke, how much is 4 times 3, how old is China and give me a riddle

Repository

https://github.com/morsh/multilingual-uber-bot

Mor Shemesh

Follow