Collecting Form Data with a LUIS-Based Bot

Ami Turgman

Background

One major advantage of chatbots, over traditional form-based applications, is that they can extract information from natural language and reduce the amount of time spent on repeat typing and entry of information. For example, if a user wants to order a medium mushroom pizza to their house at 4:00 pm, they could just type “Can I have a medium mushroom pizza sent to me at 4:00 pm” and the bot will understand what the user wants.

In order to provide this functionality, chatbots must be able to process natural language queries and convert user intents and entities into actions. For example, a user might ask the bot: “What is the weather in Tel Aviv?” The chatbot must then be able to understand the user wants to know the weather in the city of Tel Aviv, and map this information to a function such as getWeather(City). This function then takes a City parameter like Tel Aviv as input and returns the weather in Tel Aviv to the user. In this case, getting the weather is the intent and Tel Aviv is the location entity.

Chatbots often rely on language understanding services such as Microsoft LUIS to determine intents and entities. While these services are great at identifying intents and good at extracting associated entities, they are unable to guarantee that they will identify all the entities from a user’s query that are needed to perform an operation associated with a given intent. Additionally, even when services such as LUIS correctly identify all the provided associated entities in a given query, the user still might not have provided all the information required to process their request. In these cases, it is important to have a mechanism that continues to query a user in an unobtrusive manner until all the required information is gathered.

The Engagement

We recently utilized these insights about chatbots during an engagement with a partner, moed.ai, that offers a configuration-based platform for scheduling services. Their customers use moed.ai’s designated dashboard to manage the availability of their resources independently. We worked with moed.ai to create a scheduling assistance bot that their customers could then embed on their websites or Facebook pages. To read more about our work with moed.ai, please refer to a previous code story on this blog that describes how we created a single bot service to serve all of their customers.

This code story will specifically detail how we optimized the accuracy of the intent and entity extraction process for moed.ai’s scheduling assistance bot. We wanted to create a flow that interacts with a user to collect any missing information before processing their request. Our solution prompts the user for additional data related to the intent when it is not already provided and cannot be resolved automatically by LUIS.

The Solution

The success of an entity extraction process depends on the following factors:

Training the Model

The more annotated utterances (text samples) you provide to the model, the more accurate it will be in resolving the user intent and extracting the relevant entities. This increased accuracy from training the model occurs because the algorithm executed on the user’s input is based on insights it generates from these samples.

Partial User-Provided Details

One possible scenario for the scheduling assistance bot we created is booking test drives at a car dealership. When a user requests a test drive, LUIS can probably identify the user intent, and (depending on what details the user provides) the entities. For example, if a user says “I want to schedule a test drive of a SEAT Leon for tomorrow,” LUIS will probably extract the intent of test driving a vehicle, and the entities of car type (SEAT Leon) and time (tomorrow). If the user only says “I want to schedule a test drive,” LUIS can still identify the intent, but we will need to figure out the critical related entities such as car type, time, etc. We can collect this missing data that is necessary to schedule a test drive by questioning the user directly before processing the scenario.

The Code

A sample bot app is available on GitHub. You can use this example as a reference for creating your own solution.

Configuration

We wanted our solution to be configuration-based so that we could define the supported intents and which entities needed to be provided to process each of them:

For example, in the following configuration, we describe schedule and cancel intents, and the entities are service, time, carType, name, phone and age. Each intent defines a list of entities that are necessary for a successful processing.

Examples of requests a user might make:

• I would like to schedule a test drive for SEAT Leon for Sunday.
• Hi, my name is Ami, and I would like to schedule a test drive.

{
  "intents": {
    "schedule": ["service", "time", "carType", "name", "phone", "age"],
    "cancel": ["service", "time", "name"]
  },
  "fields": {
    "name": {
      "type": "text",
      "prompt": "what's your name?"
    },
    "service": {
      "type": "choice",
      "prompt": "select a service:",
      "options": [ 
        {
          "title": "test drive", 
          "value": "testDrive"
        },
        {
          "title": "cancel an appoinment", 
          "value": "cancel"
        }
      ]
    },
    .
    .
    .
  }
}

Main Loop Dialog

This dialog asks the user to provide their request, then calls on LUIS to extract the user’s intent and available entities:

// main loop- ask for text, process intent and start over
bot.dialog('/loop', [

  // ask for input
  session => {
    builder.Prompts.text(session, 'How can I help?');
  },

  // get text, call LUIS to extract intent and entities, 
  // prompt user to fill in missing fields
  (session, args, next) => {

    return luis.query(args.response).then(luisResult => {
      const intent = luisResult.topScoringIntent.intent;

      // prepare fields list based on intent (get from configuration)
      var form = prepareForm(intent);
  
      // update field with LUIS result if available
      form.forEach(field => {
        luisResult.entities.forEach(entity => {
          if (entity.type === field.name) {
            field.luisEntity = entity;
          }
        });
      });

      // collect missing fields
      return session.beginDialog('/collectFormData', { form });
    })
  },

  // get filled-in form and process the request
  (session, args) => {

    var form = args.form;
    session.send('processing request..');

    // now that we have the filled in form with all fields populated either 
    // from LUIS or the user, we can implement the request processing here...
    // TODO: process intent using the filled in form object

    // start over
    session.send('bye bye');
    return session.replaceDialog('/loop');
  }
]);

CollectFormData Dialog

This dialog retrieves a list of fields to fill in like you would with a form. Then, it attempts to determine a value for each field based on the entities extracted by LUIS from the user input. If it is unable, it prompts the user to enter a value for that field.

// collect form data dialog
bot.dialog('/collectFormData', [

  // prompt for field value
  (session, args, next) => {

    const form = args.form;
    session.dialogData.form = form;
    
    // iterate through the form fields and pick the first one without a value
    for (var i=0; i< form.length; i++) {
      var field = form[i];
      session.dialogData.fieldIndex = i;
      
      if (!field.value) {
        // if this was resolved by LUIS, move to the next handler to process the value
        if (field.luisEntity) return next();

        // prompt for value for this field
        return builder.Prompts.text(session, field.prompt, options);
      }
    }

    return session.endDialogWithResult({ form: session.dialogData.form });
  },

  // process input (either by LUIS or the user)
  (session, result, next) => {

    var field = session.dialogData.form[session.dialogData.fieldIndex];

    switch (field.type) {

      case 'number':
        // implement parsing a number from user's input or LUIS
        //break;

      case 'choice':
        // implement parsing a selection from user's input or LUIS
        //break;

      case 'time':
        // implement parsing a time from user's input or LUIS
        //break;

      // text
      default: 
        field.value = field.luisEntity ? field.luisEntity.entity : result.response;
        
    }
    
    delete field.luisEntity;

    // call collectFormData again to continue to the next field
    session.replaceDialog('/collectFormData', { form: session.dialogData.form });
  }
]);

Suggestions

If you are designing a similar system, it might be a good idea to provide options for the user to choose from in the event that LUIS doesn’t recognize the intent correctly. In this case, we would switch from the free text approach to a closed approach. We would then guide the user through the available options, asking them to choose what they would like to do, and then prompting them to provide the extra information required to process his request, the same way we did before.

Opportunities for Reuse

For our scenario with moed.ai, our solution helps ensure their customer’s requests are understood and scheduled correctly without a need for additional intervention. This approach could be reused in other scenarios where LUIS is being utilized to extract intents and entities, and it is necessary to guarantee that all form fields are filled out before processing a request. For instance, an example scenario might be a pizza ordering bot, which needs to gather information like the user’s address, toppings, drinks, etc. before submitting their order.

In addition, this approach could be extended to support more data types, such as media files like images or videos, that might be useful in processing a user’s intent. Validation logics could also be implemented as part of this flow, to verify the input before moving on the next field.

Ami Turgman

Follow