{"id":2118,"date":"2017-02-14T02:08:04","date_gmt":"2017-02-14T10:08:04","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/index.php\/2017\/02\/14\/collecting-form-data-with-a-luis-based-bot\/"},"modified":"2020-03-19T10:19:34","modified_gmt":"2020-03-19T17:19:34","slug":"collecting-form-data-with-a-luis-based-bot","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/collecting-form-data-with-a-luis-based-bot\/","title":{"rendered":"Collecting Form Data with a LUIS-Based Bot"},"content":{"rendered":"

Background<\/h2>\n

One major advantage of chatbots, over traditional form-based applications, is that they can extract information from natural language and reduce the amount of time spent on repeat typing and entry of information. For example, if a user wants to order a medium mushroom pizza to their house at 4:00 pm, they could just type \u201cCan I have a medium mushroom pizza sent to me at 4:00 pm\u201d and the bot will understand what the user wants.<\/p>\n

In order to provide this functionality, chatbots must be able to process natural language queries and convert user intents and entities into actions. For example, a user might ask the bot: \u201cWhat is the weather in Tel Aviv?\u201d The chatbot must then be able to understand the user wants to know the weather in the city of Tel Aviv, and map this information to a function such as getWeather(City)<\/code>. This function then takes a City<\/code> parameter like Tel Aviv<\/code> as input and returns the weather in Tel Aviv to the user. In this case, getting the weather<\/em> is the intent<\/strong> and Tel Aviv<\/em> is the location entity<\/strong>.<\/p>\n

Chatbots often rely on language understanding services such as Microsoft LUIS<\/a> to determine intents and entities. While these services are great at identifying intents and good at extracting associated entities, they are unable to guarantee that they will identify all the entities from a user\u2019s query that are needed to perform an operation associated with a given intent. Additionally, even when services such as LUIS correctly identify all the provided associated entities in a given query, the user still might not have provided all the information required to process their request. In these cases, it is important to have a mechanism that continues to query a user in an unobtrusive manner until all the required information is gathered.<\/p>\n

The Engagement<\/h2>\n

We recently utilized these insights about chatbots during an engagement with a partner, moed.ai<\/a>, that offers a configuration-based platform for scheduling services. Their customers use moed.ai\u2019s designated dashboard to manage the availability of their resources independently. We worked with moed.ai to create a scheduling assistance bot that their customers could then embed on their websites or Facebook pages. To read more about our work with moed.ai, please refer to a previous code story<\/a> on this blog that describes how we created a single bot service to serve all of their customers.<\/p>\n

This code story will specifically detail how we optimized the accuracy of the intent and entity extraction process for moed.ai\u2019s scheduling assistance bot. We wanted to create a flow that interacts with a user to collect any missing information before processing their request. Our solution prompts the user for additional data related to the intent when it is not already provided and cannot be resolved automatically by LUIS.<\/p>\n

The Solution<\/h2>\n

The success of an entity extraction process depends on the following factors:<\/p>\n

Training the Model<\/h3>\n

The more annotated utterances (text samples) you provide to the model, the more accurate it will be in resolving the user intent and extracting the relevant entities. This increased accuracy from training the model occurs because the algorithm executed on the user\u2019s input is based on insights it generates from these samples.<\/p>\n

Partial User-Provided Details<\/h3>\n

One possible scenario for the scheduling assistance bot we created is booking test drives at a car dealership. When a user requests a test drive, LUIS can probably identify the user intent, and (depending on what details the user provides) the entities. For example, if a user says \u201cI want to schedule a test drive of a SEAT Leon for tomorrow,\u201d LUIS will probably extract the intent of test driving a vehicle, and the entities of car type (SEAT Leon<\/code>) and time (tomorrow<\/code>). If the user only says \u201cI want to schedule a test drive,\u201d LUIS can still identify the intent, but we will need to figure out the critical related entities such as car type, time, etc. We can collect this missing data that is necessary to schedule a test drive by questioning the user directly before processing the scenario.<\/p>\n

The Code<\/h2>\n

A sample bot app<\/a> is available on GitHub. You can use this example as a reference for creating your own solution.<\/p>\n

Configuration<\/h3>\n

We wanted our solution to be configuration-based so that we could define the supported intents and which entities needed to be provided to process each of them:<\/p>\n

For example, in the following configuration, we describe schedule<\/code> and cancel<\/code> intents, and the entities are service<\/code>, time<\/code>, carType<\/code>, name<\/code>, phone<\/code> and age<\/code>. Each intent defines a list of entities that are necessary for a successful processing.<\/p>\n

Examples of requests a user might make:<\/p>\n

    \n
  • I would like to schedule a test drive for SEAT Leon for Sunday.<\/code><\/li>\n
  • Hi, my name is Ami, and I would like to schedule a test drive.<\/code><\/li>\n<\/ul>\n
    \n
    {<\/span>\r\n  \"intents\"<\/span>:<\/span> {<\/span>\r\n    \"schedule\"<\/span>:<\/span> [<\/span>\"service\"<\/span>,<\/span> \"time\"<\/span>,<\/span> \"carType\"<\/span>,<\/span> \"name\"<\/span>,<\/span> \"phone\"<\/span>,<\/span> \"age\"<\/span>],<\/span>\r\n    \"cancel\"<\/span>:<\/span> [<\/span>\"service\"<\/span>,<\/span> \"time\"<\/span>,<\/span> \"name\"<\/span>]<\/span>\r\n  },<\/span>\r\n  \"fields\"<\/span>:<\/span> {<\/span>\r\n    \"name\"<\/span>:<\/span> {<\/span>\r\n      \"type\"<\/span>:<\/span> \"text\"<\/span>,<\/span>\r\n      \"prompt\"<\/span>:<\/span> \"what's your name?\"<\/span>\r\n    },<\/span>\r\n    \"service\"<\/span>:<\/span> {<\/span>\r\n      \"type\"<\/span>:<\/span> \"choice\"<\/span>,<\/span>\r\n      \"prompt\"<\/span>:<\/span> \"select a service:\"<\/span>,<\/span>\r\n      \"options\"<\/span>:<\/span> [<\/span> \r\n        {<\/span>\r\n          \"title\"<\/span>:<\/span> \"test drive\"<\/span>,<\/span> \r\n          \"value\"<\/span>:<\/span> \"testDrive\"<\/span>\r\n        },<\/span>\r\n        {<\/span>\r\n          \"title\"<\/span>:<\/span> \"cancel an appoinment\"<\/span>,<\/span> \r\n          \"value\"<\/span>:<\/span> \"cancel\"<\/span>\r\n        }<\/span>\r\n      ]<\/span>\r\n    },<\/span>\r\n    .<\/span>\r\n    .<\/span>\r\n    .<\/span>\r\n  }<\/span>\r\n}<\/span>\r\n<\/code><\/pre>\n<\/div>\n
    Main Loop Dialog<\/h3>\n
    This dialog asks the user to provide their request, then calls on LUIS to extract the user\u2019s intent and available entities:<\/p>\n
    \n
    \r\n\/\/ main loop- ask for text, process intent and start over<\/span>\r\nbot<\/span>.<\/span>dialog<\/span>(<\/span>'\/loop'<\/span>,<\/span> [<\/span>\r\n\r\n  \/\/ ask for input<\/span>\r\n  session<\/span> =><\/span> {<\/span>\r\n    builder<\/span>.<\/span>Prompts<\/span>.<\/span>text<\/span>(<\/span>session<\/span>,<\/span> 'How can I help?'<\/span>);<\/span>\r\n  },<\/span>\r\n\r\n  \/\/ get text, call LUIS to extract intent and entities, <\/span>\r\n  \/\/ prompt user to fill in missing fields<\/span>\r\n  (<\/span>session<\/span>,<\/span> args<\/span>,<\/span> next<\/span>)<\/span> =><\/span> {<\/span>\r\n\r\n    return<\/span> luis<\/span>.<\/span>query<\/span>(<\/span>args<\/span>.<\/span>response<\/span>).<\/span>then<\/span>(<\/span>luisResult<\/span> =><\/span> {<\/span>\r\n      const<\/span> intent<\/span> =<\/span> luisResult<\/span>.<\/span>topScoringIntent<\/span>.<\/span>intent<\/span>;<\/span>\r\n\r\n      \/\/ prepare fields list based on intent (get from configuration)<\/span>\r\n      var<\/span> form<\/span> =<\/span> prepareForm<\/span>(<\/span>intent<\/span>);<\/span>\r\n  \r\n      \/\/ update field with LUIS result if available<\/span>\r\n      form<\/span>.<\/span>forEach<\/span>(<\/span>field<\/span> =><\/span> {<\/span>\r\n        luisResult<\/span>.<\/span>entities<\/span>.<\/span>forEach<\/span>(<\/span>entity<\/span> =><\/span> {<\/span>\r\n          if<\/span> (<\/span>entity<\/span>.<\/span>type<\/span> ===<\/span> field<\/span>.<\/span>name<\/span>)<\/span> {<\/span>\r\n            field<\/span>.<\/span>luisEntity<\/span> =<\/span> entity<\/span>;<\/span>\r\n          }<\/span>\r\n        });<\/span>\r\n      });<\/span>\r\n\r\n      \/\/ collect missing fields<\/span>\r\n      return<\/span> session<\/span>.<\/span>beginDialog<\/span>(<\/span>'\/collectFormData'<\/span>,<\/span> {<\/span> form<\/span> });<\/span>\r\n    })<\/span>\r\n  },<\/span>\r\n\r\n  \/\/ get filled-in form and process the request<\/span>\r\n  (<\/span>session<\/span>,<\/span> args<\/span>)<\/span> =><\/span> {<\/span>\r\n\r\n    var<\/span> form<\/span> =<\/span> args<\/span>.<\/span>form<\/span>;<\/span>\r\n    session<\/span>.<\/span>send<\/span>(<\/span>'processing request..'<\/span>);<\/span>\r\n\r\n    \/\/ now that we have the filled in form with all fields populated either <\/span>\r\n    \/\/ from LUIS or the user, we can implement the request processing here...<\/span>\r\n    \/\/ TODO: process intent using the filled in form object<\/span>\r\n\r\n    \/\/ start over<\/span>\r\n    session<\/span>.<\/span>send<\/span>(<\/span>'bye bye'<\/span>);<\/span>\r\n    return<\/span> session<\/span>.<\/span>replaceDialog<\/span>(<\/span>'\/loop'<\/span>);<\/span>\r\n  }<\/span>\r\n]);<\/span>\r\n\r\n<\/code><\/pre>\n<\/div>\n
    CollectFormData Dialog<\/h3>\n
    This dialog retrieves a list of fields to fill in like you would with a form. Then, it attempts to determine a value for each field based on the entities extracted by LUIS from the user input. If it is unable, it prompts the user to enter a value for that field.<\/p>\n
    \n
    \/\/ collect form data dialog<\/span>\r\nbot<\/span>.<\/span>dialog<\/span>(<\/span>'\/collectFormData'<\/span>,<\/span> [<\/span>\r\n\r\n  \/\/ prompt for field value<\/span>\r\n  (<\/span>session<\/span>,<\/span> args<\/span>,<\/span> next<\/span>)<\/span> =><\/span> {<\/span>\r\n\r\n    const<\/span> form<\/span> =<\/span> args<\/span>.<\/span>form<\/span>;<\/span>\r\n    session<\/span>.<\/span>dialogData<\/span>.<\/span>form<\/span> =<\/span> form<\/span>;<\/span>\r\n    \r\n    \/\/ iterate through the form fields and pick the first one without a value<\/span>\r\n    for<\/span> (<\/span>var<\/span> i<\/span>=<\/span>0<\/span>;<\/span> i<\/span><<\/span> form<\/span>.<\/span>length<\/span>;<\/span> i<\/span>++<\/span>)<\/span> {<\/span>\r\n      var<\/span> field<\/span> =<\/span> form<\/span>[<\/span>i<\/span>];<\/span>\r\n      session<\/span>.<\/span>dialogData<\/span>.<\/span>fieldIndex<\/span> =<\/span> i<\/span>;<\/span>\r\n      \r\n      if<\/span> (<\/span>!<\/span>field<\/span>.<\/span>value<\/span>)<\/span> {<\/span>\r\n        \/\/ if this was resolved by LUIS, move to the next handler to process the value<\/span>\r\n        if<\/span> (<\/span>field<\/span>.<\/span>luisEntity<\/span>)<\/span> return<\/span> next<\/span>();<\/span>\r\n\r\n        \/\/ prompt for value for this field<\/span>\r\n        return<\/span> builder<\/span>.<\/span>Prompts<\/span>.<\/span>text<\/span>(<\/span>session<\/span>,<\/span> field<\/span>.<\/span>prompt<\/span>,<\/span> options<\/span>);<\/span>\r\n      }<\/span>\r\n    }<\/span>\r\n\r\n    return<\/span> session<\/span>.<\/span>endDialogWithResult<\/span>({<\/span> form<\/span>:<\/span> session<\/span>.<\/span>dialogData<\/span>.<\/span>form<\/span> });<\/span>\r\n  },<\/span>\r\n\r\n  \/\/ process input (either by LUIS or the user)<\/span>\r\n  (<\/span>session<\/span>,<\/span> result<\/span>,<\/span> next<\/span>)<\/span> =><\/span> {<\/span>\r\n\r\n    var<\/span> field<\/span> =<\/span> session<\/span>.<\/span>dialogData<\/span>.<\/span>form<\/span>[<\/span>session<\/span>.<\/span>dialogData<\/span>.<\/span>fieldIndex<\/span>];<\/span>\r\n\r\n    switch<\/span> (<\/span>field<\/span>.<\/span>type<\/span>)<\/span> {<\/span>\r\n\r\n      case<\/span> 'number'<\/span>:<\/span>\r\n        \/\/ implement parsing a number from user's input or LUIS<\/span>\r\n        \/\/break;<\/span>\r\n\r\n      case<\/span> 'choice'<\/span>:<\/span>\r\n        \/\/ implement parsing a selection from user's input or LUIS<\/span>\r\n        \/\/break;<\/span>\r\n\r\n      case<\/span> 'time'<\/span>:<\/span>\r\n        \/\/ implement parsing a time from user's input or LUIS<\/span>\r\n        \/\/break;<\/span>\r\n\r\n      \/\/ text<\/span>\r\n      default<\/span>:<\/span> \r\n        field<\/span>.<\/span>value<\/span> =<\/span> field<\/span>.<\/span>luisEntity<\/span> ?<\/span> field<\/span>.<\/span>luisEntity<\/span>.<\/span>entity<\/span> :<\/span> result<\/span>.<\/span>response<\/span>;<\/span>\r\n        \r\n    }<\/span>\r\n    \r\n    delete<\/span> field<\/span>.<\/span>luisEntity<\/span>;<\/span>\r\n\r\n    \/\/ call collectFormData again to continue to the next field<\/span>\r\n    session<\/span>.<\/span>replaceDialog<\/span>(<\/span>'\/collectFormData'<\/span>,<\/span> {<\/span> form<\/span>:<\/span> session<\/span>.<\/span>dialogData<\/span>.<\/span>form<\/span> });<\/span>\r\n  }<\/span>\r\n]);<\/span>\r\n\r\n<\/code><\/pre>\n<\/div>\n
    Suggestions<\/h3>\n
    If you are designing a similar system, it might be a good idea to provide options for the user to choose from in the event that LUIS doesn\u2019t recognize the intent correctly. In this case, we would switch from the free text<\/em> approach to a closed<\/em> approach. We would then guide the user through the available options, asking them to choose what they would like to do, and then prompting them to provide the extra information required to process his request, the same way we did before.<\/p>\n
    Opportunities for Reuse<\/h2>\n
    For our scenario with moed.ai, our solution helps ensure their customer\u2019s requests are understood and scheduled correctly without a need for additional intervention. This approach could be reused in other scenarios where LUIS is being utilized to extract intents and entities, and it is necessary to guarantee that all form fields are filled out before processing a request. For instance, an example scenario might be a pizza ordering bot, which needs to gather information like the user\u2019s address, toppings, drinks, etc. before submitting their order.<\/p>\n
    In addition, this approach could be extended to support more data types, such as media files like images or videos, that might be useful in processing a user\u2019s intent. Validation logics could also be implemented as part of this flow, to verify the input before moving on the next field.<\/p>\n","protected":false},"excerpt":{"rendered":"
    This post outlines the steps of creating a bot that interacts with a user to collect missing LUIS entity data before processing their request.<\/p>\n","protected":false},"author":21349,"featured_media":2295,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[13],"tags":[110,231,261],"class_list":["post-2118","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bots","tag-bots","tag-language-understanding-intelligent-service-luis","tag-moed-ai"],"acf":[],"blog_post_summary":"
    This post outlines the steps of creating a bot that interacts with a user to collect missing LUIS entity data before processing their request.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/users\/21349"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/comments?post=2118"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2118\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media?parent=2118"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/categories?post=2118"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/tags?post=2118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}