Overview

We recently worked with Ovamba to build a bot that could simplify their funding application intake process. This code story explores the modular, reusable bot architecture we built, the localization we implemented, and unique challenges we faced while creating the bot (for example, working with the new Adaptive Cards, and combining Webpack and Bot Framework). Below you’ll learn about our bot-builder template, which can speed up your own bot development, and how we solved some common challenges regarding integrating other technologies with the Bot Framework.

Background

Ovamba is a USA-based startup focused on helping Africa’s small and medium enterprises (SMEs) with short-term trade and growth capital funding. Given that the majority of their customers are currently using feature phones (not smartphones), Ovamba wanted a solution to facilitate their funding application process with new customers that would run great on a feature phone.

Using the Microsoft Bot Framework (MBF), we were able to build a bot that could tap into a multitude of channels, including Ovamba’s existing Android application, to meet their customers on their preferred conversation platforms.

Bot Framework

Bots enable you to communicate with users through conversation; these conversations can take place through text, speech, cards, buttons, and other visuals. The advantage of building a bot is that it enables you to reach users through their preferred method of communication with a single codebase.

Microsoft Bot Framework is a platform that enables you to build bots using the C# or Node.js SDKs. Essentially, when you build a bot, you are building a RESTful web service. The Bot Framework service enables you to easily connect to channels such as Facebook, Skype, Slack, etc. If you’d like to learn more about the Bot Framework, the documentation provides great information.

Because the Bot Framework can be localized to any language and connected to channels like SMS, Facebook Messenger, and mobile apps, Ovamba decided that a bot would be the best avenue to help them to communicate with, and gather information from their customers. The Microsoft Bot Framework was a good fit because it provides all these features, in particular:

Localization

The localization feature enables users to communicate with the bot in multiple languages. One of Ovamba’s top feature requirements was the ability to communicate with each customer in his or her local dialect.

Android & SMS Availability

Although we didn’t focus on SMS at this hack, SMS and Android support was a top priority for Ovamba, since most of their customers use either Android or simple feature phones that don’t run apps. One useful feature of the Bot Framework is support for SMS as a channel; no matter how complex and visual your bot is, the communication is stripped down so that it works well on SMS, too.

Speech

In addition to text-based communication, the Bot Framework enables a user to speak to a bot and have the bot to respond back in speech. In some cases, you may have to provide your own speech to text support on the client based on the channel you’re using (for example, Android). One way to accomplish this task is to record what the user says, translate it (on the device or in the cloud), then send the text to the bot. Alternatively, the Bot Framework has a third party NodeJS npm package, botbuilder-calling, that enables similar functionality.

Bot Architecture

To get a simple, modular, and testable architecture, we merged two bot templates: SWELL Generator and transmute-bot. This combination became the building block for the hackfest. The resulting architecture includes scripts for automated building, packaging of our release code, testing (using mochajs, chaijs, and sinonjs), and visual code coverage.

The modularity came in the form of what we called “Skills”. A skill encapsulates all the code for your Dialogs, Messages, Intents, and other capabilities. Skills enable you to break down the functionality of your bot into modular pieces.

Once you’ve created a skill, simply register it with the bot to give the bot those capabilities and dialogs.

bot.ts:

private registerDialogs() {
    // this.bot.dialog('/', this.dialogs);
    RootSkill.register(this.bot, this.dialogs);
    StartSkill.register(this.bot, this.dialogs);
    MainMenuSkill.register(this.bot, this.dialogs); // Add a line like this for every dialog
    BasicInfoFormSkill.register(this.bot, this.dialogs);
    LanguageSelectionSkill.register(this.bot, this.dialogs);
}

private bindDialogsToIntents() {
    this.dialogs.matches('start', StartSkill.Dialogs.Start);
    this.dialogs.matches('mainMenu', MainMenuSkill.Dialogs.MainMenu); // Add a line like this for every intent
    this.dialogs.matches('basicInfoForm', BasicInfoFormSkill.Dialogs.BasicInfoForm);
    this.dialogs.matches('languageSelection', LanguageSelectionSkill.Dialogs.LanguageSelection);
}

You can find our botbuilder template on GitHub.

Localization

Ultimately, Ovamba’s goal is to localize their applications and services into their customers’ regional African dialects. Given the current lack of translations from English to these lesser-known dialects, we focused on an initial set of 3 languages: English, French, and Arabic. The full-featured Bot Framework has support for localization built in:

By default, the localization system will search for the bot’s prompts in the ./locale/{lang-tag}/index.json file where {lang-tag} is a valid IETF language tag representing the preferred locale for which to find prompts.

To accomplish this task, we created a locale  folder in the bot and defined sub-folders of ISO language codes (e.g. locale/fr) and included our index.json  file there (locale/fr/index.json).

{ 
   .
   .
   .

   "B-CNPS": "Quel est votre numéro CNPS?",
   "B-DrivingLicense": "Quel est votre numéro de permis de conduire?",
   "B-NationalId": "Quel est votre numéro d'identification nationale?",
   "B-LanguagesSpoken": "Quelles langues connaissez-vous?",
   "B-Occupation": "Quel est votre occupation?",
   "B-Address": "Quelle est la première ligne de votre adresse?",
   "B-City": "Dans quelle ville vis-vous?",
   .
   .
   .
}

Now you can use the session.localizer.gettext(session.preferredLocale(), “B-CNPS”)   and the Bot Framework will automatically retrieve the correct localized string for you. Note that if session.preferredLocale() isn’t set, the Bot Framework will default to the English locale.

Localization Issues

Since we were using dependencies (botbuilder-forms and adaptive cards) that did not directly support localization, we had to add that support explicitly. For adaptive cards, we wrote a helper method that converts specific text found in the JSON schema of adaptive cards to the localized versions at run-time:

export function localizeAdaptiveCards(cardQuestions: any, session: Session) {
    for (const q of cardQuestions.body) {
        q.text = (q.hasOwnProperty("text")) ? session.localizer.gettext(session.preferredLocale(), q.text) : null;
        q.speak = (q.hasOwnProperty("speak")) ? session.localizer.gettext(session.preferredLocale(), q.speak) : null;
        q.placeholder = (q.hasOwnProperty("placeholder")) ? session.localizer.gettext(session.preferredLocale(), q.placeholder) : null;
    }
    for (const a of cardQuestions.actions) {
        a.title = session.localizer.gettext(session.preferredLocale(), a.title);
    }
}

We added support for localization to BotBuilder-Forms by appropriately calling session.localizer.gettext(session.preferredLocale(), “sometext”)  where needed. These updates were submitted as a pull request back to the open source project.

Challenges

Localization

We ran into several interesting issues while adding localization support. The first was directly related to how the Bot Framework resolves file names for the localization choice. This problem is described in Issue #2 – Botbuilder Framework, which is still open at the time of writing. In bot.ts , the following code loads the localization JSON files from the proper directory.

const botLocalePath = __dirname + '/locale';

private setUp() {
  const url = config.luis.url.replace('##APP##', config.luis.app).replace('##KEY##', config.luis.key);
  this.recognizer = new builder.LuisRecognizer(url);
  this.dialogs = new builder.IntentDialog({ recognizers: [this.recognizer] });

  this.bot.set(localizerSettings, {
    botLocalePath: botLocalePath, //config.languages.path,
    defaultLocale: 'en'//languageChoices[languageChoice]
  });
}

The constant botLocalePath in the first block must be set to the NodeJS internal variable of __dirname, concatenated to the path of the locale files. The __dirname value is the directory of the startup script at runtime, so the concatenation generates an absolute path. In the setUp function, the localizerSettings are initialized with the path based on the botLocalePath .

WebPack

The second issue is related to the first.  When you use WebPack for bundling and optimization, the __dirname variable is optimized away unless you make an appropriate configuration setting. If the __dirname variable is optimized away, then the code above will fail. This issue applies generally to any NodeJS application using __dirname  or __filename  bundling with WebPack. To prevent this optimization, go to the webpack.config.js  file, and ensure that the node  property contains an object with the { __dirname: false, __filename: false } , which will prevent both variables from being optimized out and should appear as is in the optimized bundle.

module.exports = {
    target: "node",
    entry: "./release/server.js",
    output: {
        path: __dirname + "/release/",
        filename: "bundle.js",
        library: '',
        libraryTarget: 'commonjs-module'
    },
    node: {
        __dirname: false,
        __filename: false
    }
};

Adaptive Cards

A third issue faced was with the new adaptive cards features in the Bot Framework, using the JSON template. The issue was that data from input fields were not being picked up. Adaptive cards enable you to create a form with a submit action, which looks something like this:

{
  "$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
  "type": "AdaptiveCard",
  "version": "1.0",
  "body": [
    {
        "type": "TextBlock",
        "text": "Your name",
        "wrap": true
    },
    {
        "type": "Input.Text",
        "id": "myName",
        "placeholder": "Last, First"
    },
    {
        "type": "TextBlock",
        "text": "Your email",
        "wrap": true
    },
    {
        "type": "Input.Text",
        "id": "myEmail",
        "placeholder": "youremail@example.com",
        "style": "email"
    },
  ],
  "actions": [
    {
      "type": "Action.Submit",
      "title": "Submit"
    }
  ]
}

Upon pressing the submit button, it will gather up the input fields and merge them with the optional data field in the submit action. It then generates an event to the client asking for data to be submitted; it’s then completely up to the client to determine how that data is processed. In the Bot Framework, the data is sent as an activity through the message endpoint. We wrote some middleware to pick this up and handle the submission of the data to Ovamba’s backend.

if (this.bot) {
    this.bot.use(
        this.routingMiddleware()
    );
}

public routingMiddleware() {
    return {
        botbuilder: (session: any, next: Function) => {
            // value contains the data from the adaptive card form
            if (session.message.value) {
                const formData = session.message.value;

                request.post(
                'OVAMBA_ENDPOINT_URL',
                {
                    json: formData
                },

                .
                .
                .

But the input field data was not getting picked up. Turns out there is an issue using  ColumnSet  (like in the code snippet below), that prevents the data from those input fields from being picked up. This issue has been raised on the project’s GitHub repo. It was an interesting bug because our code for the adaptive card worked in the browser (for example, this form) but in our bot, the data from input fields wasn’t getting passed through.

"body": [
    {
      "type": "ColumnSet",
      "columns": [
        {
          "type": "Column",
          "size": 1,
          "items": [
            {
              "type": "Input.Text",
              "id": "myName",
              "placeholder": "Last, First"
            },

Again, the issue is related to using ColumnSet, so we worked around the issue by removing the columns.

 "body": [
    {
      "type": "TextBlock",
      "text": "B-DOB",
      "speak": "B-DOB",
      "wrap": true
    },
    {
      "type": "Input.Date",
      "id": "basic_dateofbirth"
    },

Conclusion

This project with Ovamba has given us a solid template for future Bot Framework projects that includes automated scripts for building, packaging, and testing code. We’ve also built solutions to localize adaptive cards and botbuilder-forms. This knowledge can be reused in similar projects that require localization and adaptive cards or turn-based prompts.

Future Work

We plan to continue our work with Ovamba, especially building an African dialect language recognition system to enable application localization and cater to the specific dialects of their customers. This task presents many interesting challenges. For example, during the hackfest, we learned that some African languages we wanted to support existed in spoken form only (no written form), which left us considering how might we localize text for these languages. We look forward to tackling these challenges and more in our future work with Ovamba.