{"id":2139,"date":"2016-09-12T17:00:00","date_gmt":"2016-09-12T17:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/reallifecode\/index.php\/2016\/09\/12\/servicing-faq-style-content-through-a-conversational-interface\/"},"modified":"2020-03-15T07:09:44","modified_gmt":"2020-03-15T14:09:44","slug":"servicing-faq-style-content-through-a-conversational-interface","status":"publish","type":"post","link":"https:\/\/devblogs.microsoft.com\/ise\/servicing-faq-style-content-through-a-conversational-interface\/","title":{"rendered":"Servicing FAQ-Style Content Through a Conversational Interface"},"content":{"rendered":"<h2 id=\"background\">Background<\/h2>\n<p>Recent advances in Machine Learning coupled with the growing ubiquity of instant messaging apps have revived interest in the decades-old concept of chatbots, now more commonly known as \u2018bots\u2019.<\/p>\n<p>Microsoft has taken a leading position in this field with the release of the <a href=\"https:\/\/dev.botframework.com\/\">Bot Framework SDK<\/a> and <a href=\"https:\/\/www.luis.ai\/\">LUIS<\/a> language understanding service.<\/p>\n<p>However developing for natural language can still be daunting when approaching the field for the first time. We present below an example of integrating an existing natural language powered service in order to serve structured, FAQ-style content through a conversational interface.<\/p>\n<h2 id=\"the-problem\">The Problem<\/h2>\n<p>During a recent customer engagement we were asked to explore the possibility of publishing existing FAQ-style content via a conversational interface.<\/p>\n<p>Common with almost every consumer-facing business, our customer\u2019s call centre staff were spending significant amounts of their time answering simple queries that could otherwise have been found on the company\u2019s web site. Perhaps serving this content via a conversational interface could give customers a lower friction method to directly find the answers to their questions rather than searching a potentially long list of questions looking for a match.<\/p>\n<p>In addition, once made available through the Bot Framework, the same content can be simultaneously published to a wide variety of channels in a manner that makes detailed usage feedback almost trivial to collect.<\/p>\n<p>With only two days to complete the engagement we were concerned that a traditional approach involving building domain-specific language understanding models would prove too labour intensive. We needed to pull a rabbit out of a hat somehow. Step forward <a href=\"https:\/\/qnamaker.botframework.com\/\">QnA Maker<\/a>.<\/p>\n<h2 id=\"outline-solution\">Outline Solution<\/h2>\n<p>The <a href=\"https:\/\/qnamaker.botframework.com\/\">QnA Maker<\/a> service lets you easily create a knowledge base from existing online and editorial sources. You can tweak and train the responses and when satisfied with their quality, publish to an endpoint that can be accessed from your Bot. The service behind that endpoint takes care of all the complex natural language processing tasks needed to understand the question and returns the best answer it can find in the knowledge base.<\/p>\n<p>Calling the service is done through a super simple HTTP call. You can even call it directly from your browser:<\/p>\n<p><sub><a href=\"http:\/\/qnaservice.cloudapp.net\/KBService.svc\/GetAnswer?kbId=a3dae93561d24529b11a4c0eb8acd444&amp;question=What%20is%20cortana\">http:\/\/qnaservice.cloudapp.net\/KBService.svc\/GetAnswer?kbId=a3dae93561d24529b11a4c0eb8acd444&amp;question=What%20is%20cortana<\/a><\/sub><\/p>\n<p>This makes publishing our FAQ almost trivial from within a Bot application and the top-level architecture similarly, extremely simple;<\/p>\n<p> <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2016\/09\/TLArch.png\" alt=\"Image TLArch\" width=\"720\" height=\"405\" class=\"aligncenter size-full wp-image-11112\" srcset=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/TLArch.png 720w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/TLArch-300x169.png 300w\" sizes=\"(max-width: 720px) 100vw, 720px\" \/><\/p>\n<p>At its most basic level all we have to do is pass any message the user asks straight through to the QnA Maker service and return the response, after a little processing, directly back to the user.<\/p>\n<p>In Node, our target language for this project, this is very simple:<\/p>\n<div class=\"language-javascript highlighter-rouge\">\n<pre class=\"highlight\"><code>\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">q<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">querystring<\/span><span class=\"p\">.<\/span><span class=\"nx\">escape<\/span><span class=\"p\">(<\/span><span class=\"s1\">'What is cortana?'<\/span><span class=\"p\">);<\/span>\r\n<span class=\"c1\">\/\/ Using the request library here to simplify calling the QnA service <\/span>\r\n<span class=\"nx\">request<\/span><span class=\"p\">(<\/span><span class=\"nx\">config<\/span><span class=\"p\">.<\/span><span class=\"nx\">get<\/span><span class=\"p\">(<\/span><span class=\"s1\">'qnaUri'<\/span><span class=\"p\">)<\/span> <span class=\"o\">+<\/span> <span class=\"nx\">q<\/span><span class=\"p\">,<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">(<\/span><span class=\"nx\">error<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">,<\/span> <span class=\"nx\">body<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"nx\">session<\/span><span class=\"p\">.<\/span><span class=\"nx\">send<\/span><span class=\"p\">(<\/span><span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">answer<\/span><span class=\"p\">);<\/span> \r\n<span class=\"p\">});<\/span>\r\n\r\n<\/code><\/pre>\n<\/div>\n<p>where \u2018qnaUri\u2019 is our QnA service endpoint and \u2018q\u2019 is the verbatim, URI-escaped question.<\/p>\n<p>Of course we can add as much extra complexity to the system as we wish, additional commands and logging of user behaviour perhaps, but at it\u2019s core the solution really isn\u2019t much more complex than what we\u2019ve already shown, in fact the near-minimal implementation to just around 100 lines. Wow! That\u2019s some heavy lifting the Bot Framework and the QnA Maker are already doing for us.<\/p>\n<h2 id=\"implementation\">Implementation<\/h2>\n<p>The minimal implementation of the complete QnA bot runs to just around a hundred lines of code:<\/p>\n<div class=\"language-javascript highlighter-rouge\">\n<pre class=\"highlight\"><code><span class=\"s1\">'use strict'<\/span><span class=\"p\">;<\/span>\r\n\r\n<span class=\"cm\">\/*\r\n  qnamakerbot - Minimal demo of integrating a bot with the QnA Maker \r\n  (https:\/\/qnamaker.botframework.com\/). \r\n\r\n  Uses the following libraries:\r\n\r\n  request (https:\/\/www.npmjs.com\/package\/request) - Simplifies calling HTTP services.\r\n  querystring (https:\/\/www.npmjs.com\/package\/query-string) - Escapes text strings suitable for using in HTTP queries.\r\n  restify (https:\/\/www.npmjs.com\/package\/restify) - Framework to simplify building RESTful APIs\r\n  botbuilder (https:\/\/www.npmjs.com\/package\/botbuilder) - The Microsoft Bot Framework SDK.\r\n*\/<\/span>\r\n\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">request<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">'request'<\/span><span class=\"p\">);<\/span>\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">querystring<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">'querystring'<\/span><span class=\"p\">);<\/span>\r\n\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">restify<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">'restify'<\/span><span class=\"p\">);<\/span>\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">botbuilder<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">'botbuilder'<\/span><span class=\"p\">);<\/span>\r\n\r\n<span class=\"c1\">\/\/ Config can appear in the environment, argv or in the local config<\/span>\r\n<span class=\"c1\">\/\/ file (don't check this in!!)<\/span>\r\n<span class=\"kd\">var<\/span> <span class=\"nx\">config<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">require<\/span><span class=\"p\">(<\/span><span class=\"s1\">'nconf'<\/span><span class=\"p\">).<\/span><span class=\"nx\">env<\/span><span class=\"p\">().<\/span><span class=\"nx\">argv<\/span><span class=\"p\">().<\/span><span class=\"nx\">file<\/span><span class=\"p\">({<\/span> <span class=\"na\">file<\/span><span class=\"p\">:<\/span> <span class=\"s1\">'.\/localConfig.json'<\/span> <span class=\"p\">});<\/span>\r\n\r\n<span class=\"kd\">function<\/span> <span class=\"nx\">createChatConnector<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Create the chat connector which will hook us up to the Bot framework<\/span>\r\n  <span class=\"c1\">\/\/ servers. This is all very standard stuff.<\/span>\r\n  <span class=\"c1\">\/\/ Full details here: https:\/\/docs.botframework.com\/en-us\/node\/builder\/overview\/#navtitle<\/span>\r\n\r\n  <span class=\"kd\">let<\/span> <span class=\"nx\">opts<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"c1\">\/\/ You generate these two values when you create your bot at<\/span>\r\n    <span class=\"c1\">\/\/ https:\/\/dev.botframework.com\/<\/span>\r\n    <span class=\"na\">appId<\/span><span class=\"p\">:<\/span> <span class=\"nx\">config<\/span><span class=\"p\">.<\/span><span class=\"nx\">get<\/span><span class=\"p\">(<\/span><span class=\"s1\">'BOT_APP_ID'<\/span><span class=\"p\">),<\/span>\r\n    <span class=\"na\">appPassword<\/span><span class=\"p\">:<\/span> <span class=\"nx\">config<\/span><span class=\"p\">.<\/span><span class=\"nx\">get<\/span><span class=\"p\">(<\/span><span class=\"s1\">'BOT_APP_PASSWORD'<\/span><span class=\"p\">)<\/span>\r\n  <span class=\"p\">};<\/span>\r\n\r\n  <span class=\"kd\">let<\/span> <span class=\"nx\">chatConnector<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nx\">botbuilder<\/span><span class=\"p\">.<\/span><span class=\"nx\">ChatConnector<\/span><span class=\"p\">(<\/span><span class=\"nx\">opts<\/span><span class=\"p\">);<\/span>\r\n\r\n  <span class=\"kd\">let<\/span> <span class=\"nx\">server<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">restify<\/span><span class=\"p\">.<\/span><span class=\"nx\">createServer<\/span><span class=\"p\">();<\/span>\r\n\r\n  <span class=\"nx\">server<\/span><span class=\"p\">.<\/span><span class=\"nx\">post<\/span><span class=\"p\">(<\/span><span class=\"s1\">'\/qna'<\/span><span class=\"p\">,<\/span> <span class=\"nx\">chatConnector<\/span><span class=\"p\">.<\/span><span class=\"nx\">listen<\/span><span class=\"p\">());<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Serve the static file containing the embedded web control<\/span>\r\n  <span class=\"c1\">\/\/ directly from the local filesystem<\/span>\r\n  <span class=\"nx\">server<\/span><span class=\"p\">.<\/span><span class=\"nx\">get<\/span><span class=\"p\">(<\/span><span class=\"sr\">\/<\/span><span class=\"se\">\/?<\/span><span class=\"sr\">.*\/<\/span><span class=\"p\">,<\/span> <span class=\"nx\">restify<\/span><span class=\"p\">.<\/span><span class=\"nx\">serveStatic<\/span><span class=\"p\">({<\/span>\r\n    <span class=\"na\">directory<\/span><span class=\"p\">:<\/span> <span class=\"nx\">__dirname<\/span> <span class=\"o\">+<\/span> <span class=\"s1\">'\/html'<\/span><span class=\"p\">,<\/span>\r\n    <span class=\"na\">default<\/span><span class=\"p\">:<\/span> <span class=\"s1\">'index.html'<\/span><span class=\"p\">,<\/span>\r\n  <span class=\"p\">}));<\/span>\r\n\r\n  <span class=\"nx\">server<\/span><span class=\"p\">.<\/span><span class=\"nx\">listen<\/span><span class=\"p\">(<\/span><span class=\"mi\">3978<\/span><span class=\"p\">,<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">log<\/span><span class=\"p\">(<\/span><span class=\"s1\">'%s listening to %s'<\/span><span class=\"p\">,<\/span> <span class=\"nx\">server<\/span><span class=\"p\">.<\/span><span class=\"nx\">name<\/span><span class=\"p\">,<\/span> <span class=\"nx\">server<\/span><span class=\"p\">.<\/span><span class=\"nx\">url<\/span><span class=\"p\">);<\/span>\r\n  <span class=\"p\">});<\/span>\r\n\r\n  <span class=\"k\">return<\/span> <span class=\"nx\">chatConnector<\/span><span class=\"p\">;<\/span>\r\n<span class=\"p\">}<\/span>\r\n\r\n<span class=\"kd\">function<\/span> <span class=\"nx\">qna<\/span><span class=\"p\">(<\/span><span class=\"nx\">q<\/span><span class=\"p\">,<\/span> <span class=\"nx\">cb<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Here's where we pass anything the user typed along to the<\/span>\r\n  <span class=\"c1\">\/\/ QnA service. Super simple stuff!!<\/span>\r\n\r\n  <span class=\"nx\">q<\/span> <span class=\"o\">=<\/span> <span class=\"nx\">querystring<\/span><span class=\"p\">.<\/span><span class=\"nx\">escape<\/span><span class=\"p\">(<\/span><span class=\"nx\">q<\/span><span class=\"p\">);<\/span>\r\n  <span class=\"nx\">request<\/span><span class=\"p\">(<\/span><span class=\"nx\">config<\/span><span class=\"p\">.<\/span><span class=\"nx\">get<\/span><span class=\"p\">(<\/span><span class=\"s1\">'qnaUri'<\/span><span class=\"p\">)<\/span> <span class=\"o\">+<\/span> <span class=\"nx\">q<\/span><span class=\"p\">,<\/span> <span class=\"kd\">function<\/span> <span class=\"p\">(<\/span><span class=\"nx\">error<\/span><span class=\"p\">,<\/span> <span class=\"nx\">response<\/span><span class=\"p\">,<\/span> <span class=\"nx\">body<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n    <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">error<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n      <span class=\"nx\">cb<\/span><span class=\"p\">(<\/span><span class=\"nx\">error<\/span><span class=\"p\">,<\/span> <span class=\"kc\">null<\/span><span class=\"p\">);<\/span>\r\n    <span class=\"p\">}<\/span>\r\n    <span class=\"k\">else<\/span> <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">response<\/span><span class=\"p\">.<\/span><span class=\"nx\">statusCode<\/span> <span class=\"o\">!==<\/span> <span class=\"mi\">200<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n      <span class=\"c1\">\/\/ Valid response from QnA but it's an error<\/span>\r\n      <span class=\"c1\">\/\/ return the response for further processing<\/span>\r\n      <span class=\"nx\">cb<\/span><span class=\"p\">(<\/span><span class=\"nx\">response<\/span><span class=\"p\">,<\/span> <span class=\"kc\">null<\/span><span class=\"p\">);<\/span>\r\n    <span class=\"p\">}<\/span>\r\n    <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\r\n      <span class=\"c1\">\/\/ All looks OK, the answer is in the body<\/span>\r\n      <span class=\"nx\">cb<\/span><span class=\"p\">(<\/span><span class=\"kc\">null<\/span><span class=\"p\">,<\/span> <span class=\"nx\">body<\/span><span class=\"p\">);<\/span>\r\n    <span class=\"p\">}<\/span>\r\n  <span class=\"p\">});<\/span>\r\n<span class=\"p\">}<\/span>\r\n\r\n<span class=\"kd\">function<\/span> <span class=\"nx\">initialiseBot<\/span><span class=\"p\">(<\/span><span class=\"nx\">bot<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Someone set up us the bot :-)<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Use IntentDialog to watch for messages that match regexes<\/span>\r\n  <span class=\"kd\">let<\/span> <span class=\"nx\">intents<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nx\">botbuilder<\/span><span class=\"p\">.<\/span><span class=\"nx\">IntentDialog<\/span><span class=\"p\">();<\/span>\r\n\r\n  <span class=\"c1\">\/\/ Any message not matching the previous intents ends up here<\/span>\r\n  <span class=\"nx\">intents<\/span><span class=\"p\">.<\/span><span class=\"nx\">onDefault<\/span><span class=\"p\">((<\/span><span class=\"nx\">session<\/span><span class=\"p\">,<\/span> <span class=\"nx\">args<\/span><span class=\"p\">,<\/span> <span class=\"nx\">next<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\r\n\r\n    <span class=\"c1\">\/\/ Just throw everything into the qna service<\/span>\r\n    <span class=\"nx\">qna<\/span><span class=\"p\">(<\/span><span class=\"nx\">session<\/span><span class=\"p\">.<\/span><span class=\"nx\">message<\/span><span class=\"p\">.<\/span><span class=\"nx\">text<\/span><span class=\"p\">,<\/span> <span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">,<\/span> <span class=\"nx\">result<\/span><span class=\"p\">)<\/span> <span class=\"o\">=&gt;<\/span> <span class=\"p\">{<\/span>\r\n      <span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n        <span class=\"nx\">console<\/span><span class=\"p\">.<\/span><span class=\"nx\">error<\/span><span class=\"p\">(<\/span><span class=\"nx\">err<\/span><span class=\"p\">);<\/span>\r\n        <span class=\"nx\">session<\/span><span class=\"p\">.<\/span><span class=\"nx\">send<\/span><span class=\"p\">(<\/span><span class=\"s1\">'Unfortunately an error occurred. Try again.'<\/span><span class=\"p\">);<\/span>\r\n      <span class=\"p\">}<\/span>\r\n      <span class=\"k\">else<\/span> <span class=\"p\">{<\/span>\r\n        <span class=\"c1\">\/\/ The QnA returns a JSON: { answer:XXXX, score: XXXX: }<\/span>\r\n        <span class=\"c1\">\/\/ where score is a confidence the answer matches the question.<\/span>\r\n        <span class=\"c1\">\/\/ Advanced implementations might log lower scored questions and<\/span>\r\n        <span class=\"c1\">\/\/ answers since they tend to indicate either gaps in the FAQ content<\/span>\r\n        <span class=\"c1\">\/\/ or a model that needs training<\/span>\r\n        <span class=\"nx\">session<\/span><span class=\"p\">.<\/span><span class=\"nx\">send<\/span><span class=\"p\">(<\/span><span class=\"nx\">JSON<\/span><span class=\"p\">.<\/span><span class=\"nx\">parse<\/span><span class=\"p\">(<\/span><span class=\"nx\">result<\/span><span class=\"p\">).<\/span><span class=\"nx\">answer<\/span><span class=\"p\">);<\/span>\r\n      <span class=\"p\">}<\/span>\r\n    <span class=\"p\">});<\/span>\r\n  <span class=\"p\">});<\/span>\r\n\r\n  <span class=\"nx\">bot<\/span><span class=\"p\">.<\/span><span class=\"nx\">dialog<\/span><span class=\"p\">(<\/span><span class=\"s1\">'\/'<\/span><span class=\"p\">,<\/span> <span class=\"nx\">intents<\/span><span class=\"p\">);<\/span>\r\n<span class=\"p\">}<\/span>\r\n\r\n<span class=\"kd\">function<\/span> <span class=\"nx\">main<\/span><span class=\"p\">()<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"nx\">initialiseBot<\/span><span class=\"p\">(<\/span><span class=\"k\">new<\/span> <span class=\"nx\">botbuilder<\/span><span class=\"p\">.<\/span><span class=\"nx\">UniversalBot<\/span><span class=\"p\">(<\/span><span class=\"nx\">createChatConnector<\/span><span class=\"p\">()));<\/span>\r\n<span class=\"p\">}<\/span>\r\n\r\n<span class=\"k\">if<\/span> <span class=\"p\">(<\/span><span class=\"nx\">require<\/span><span class=\"p\">.<\/span><span class=\"nx\">main<\/span> <span class=\"o\">===<\/span> <span class=\"nx\">module<\/span><span class=\"p\">)<\/span> <span class=\"p\">{<\/span>\r\n  <span class=\"nx\">main<\/span><span class=\"p\">();<\/span>\r\n<span class=\"p\">}<\/span>\r\n<\/code><\/pre>\n<\/div>\n<h2 id=\"setting-up-the-services\">Setting up the Services<\/h2>\n<p>The complete solution requires a couple of moving parts to be up and running in order to succesfully process questions from the user.<\/p>\n<p><strong>The Bot App<\/strong> &#8211; This is the code we actually write and is responsible for receiving messages from the Bot Framework, hosted in the cloud, and passing them along to the QnA service. When the QnA Service sends us a reply our app simply checks the validity of the response, extracts the answer from the reply body and sends it, via the Bot Framework, back to the user. During development this will run on our dev machine. When we want to go to production we\u2019ll need to find a place to run this permanently in the cloud.<\/p>\n<p><strong>The Bot Framework<\/strong> &#8211; Hosted by Microsoft and running in the cloud this is the glue that links together the client applications, such as Skype, Slack, Telegram etc running on the user\u2019s machine to the Bot app running either on our dev machine or in the cloud.<\/p>\n<p><strong>The QnA Service<\/strong> &#8211; Hosted by Microsoft and running in the cloud this is the service that will receive the question the user asks from our Bot application, search its knowledge base and return the most likely answer.<\/p>\n<p><strong>ngrok<\/strong> &#8211; ngrok helps us during development by making it easy for the Bot Framework components to\nconnect to the Bot app instance running on our dev machines despite the dev machine likely sitting behind a fireweall.<\/p>\n<h3 id=\"setup-ngrok\">Setup ngrok<\/h3>\n<p>ngrok\u2019s a great tool that allows you to serve content to the public internet directly from your local machine. We\u2019ll be using it to allow the cloud-hosted components of the Bot Framework to call into the Bot application running on your dev machine.<\/p>\n<p>Download ngrok <a href=\"https:\/\/ngrok.com\/\">here<\/a>.<\/p>\n<p>Set it running:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>ngrok http 3978\r\n<\/code><\/pre>\n<\/div>\n<p>You should see something like this:\n <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2016\/09\/ngrok.png\" alt=\"Image ngrok\" width=\"1419\" height=\"483\" class=\"aligncenter size-full wp-image-11111\" srcset=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/ngrok.png 1419w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/ngrok-300x102.png 300w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/ngrok-1024x349.png 1024w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/ngrok-768x261.png 768w\" sizes=\"(max-width: 1419px) 100vw, 1419px\" \/><\/p>\n<p>The important bit here is the https forwarding address. Copy that URI (should look something like https:\/\/xxxxx.ngrok.io), we\u2019re going to be using it the next section.<\/p>\n<p>Leave ngrok running throughout the remaining steps.<\/p>\n<h3 id=\"register-your-bot\">Register Your Bot<\/h3>\n<p>Head over to the <a href=\"https:\/\/dev.botframework.com\/bots\/new\">Bot Registration Page<\/a> (MSA\/Azure Subscription required) and fill out the details of your bot. Full details can be found <a href=\"https:\/\/docs.botframework.com\/en-us\/csharp\/builder\/sdkreference\/gettingstarted.html#registering\">here<\/a>.<\/p>\n<p>Under the Configuration section, there\u2019s a section called \u2018messaging endpoint\u2019. This is how the framework is going to contact the bot app running on your machine. The ngrok URI you copied in the previous step goes in here. If you restart ngrok at any point the URI will change and you\u2019ll have to update this value. When you come to deploy your bot outside of your own machine you\u2019ll also have to come back here to update this value to the new, public address of your Bot instance.<\/p>\n<p><strong>IMPORTANT:<\/strong> When you click the button to generate your Microsoft App ID and password for your bot <strong><em>be sure you save the password somewhere safe<\/em><\/strong>. This is the only time you\u2019ll be shown this and if you lose it you\u2019ll have to go through all this setup stuff from the top again.<\/p>\n<p>With everything filled in, hit Register to create all the back-end stuff that will connect the various channels to your Bot application instance. Let\u2019s go ahead and get this hooked up to our actual bot.<\/p>\n<h3 id=\"build-the-bot\">Build the Bot<\/h3>\n<p>Clone the solution repo <a href=\"https:\/\/github.com\/CatalystCode\/qnamakerbot\/\">qnamakerbot<\/a>:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src <span class=\"nv\">$ <\/span>git clone https:\/\/github.com\/CatalystCode\/qnamakerbot.git\r\nCloning into <span class=\"s1\">'qnamakerbot'<\/span>...\r\nremote: Counting objects: 400, <span class=\"k\">done<\/span>.\r\nremote: Compressing objects: 100% <span class=\"o\">(<\/span>4\/4<span class=\"o\">)<\/span>, <span class=\"k\">done<\/span>.\r\nremote: Total 400 <span class=\"o\">(<\/span>delta 0<span class=\"o\">)<\/span>, reused 0 <span class=\"o\">(<\/span>delta 0<span class=\"o\">)<\/span>, pack-reused 396\r\nReceiving objects: 100% <span class=\"o\">(<\/span>400\/400<span class=\"o\">)<\/span>, 1.18 MiB | 2.07 MiB\/s, <span class=\"k\">done<\/span>.\r\nResolving deltas: 100% <span class=\"o\">(<\/span>232\/232<span class=\"o\">)<\/span>, <span class=\"k\">done<\/span>.\r\nChecking connectivity... <span class=\"k\">done<\/span>.\r\ntobe ~\/src <span class=\"nv\">$ <\/span><span class=\"nb\">cd <\/span>qnamakerbot\/\r\ntobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>tree\r\n.\r\n\u251c\u2500\u2500 README.md\r\n\u251c\u2500\u2500 app.js\r\n\u251c\u2500\u2500 assets\r\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 qna.png\r\n\u251c\u2500\u2500 html\r\n\u2502\u00a0\u00a0 \u2514\u2500\u2500 index.html\r\n\u251c\u2500\u2500 localConfig.json\r\n\u2514\u2500\u2500 package.json\r\n\r\n2 directories, 6 files\r\n<\/code><\/pre>\n<\/div>\n<p>A quick run through the project structure:<\/p>\n<p><strong>README.md<\/strong> &#8211; The Github readme.\n<strong>app.js<\/strong> &#8211; The entire bot app in a single file. <strong><em>All<\/em><\/strong> the code lives here.\n<strong>assets\/qna.jpg<\/strong> &#8211; Image referenced from the README file.\n<strong>html\/index.html<\/strong> &#8211; The file we\u2019re going to serve from the app that contains the webchat control through which was can talk to the Bot.\n<strong>localConfig.json<\/strong> &#8211; Contains all our configuration secrets. We\u2019ll need to fill this in later.\n<strong>package.json<\/strong> &#8211; Details about this package and its dependencies.<\/p>\n<p>Before we go any further, let\u2019s remember to install the required node packages:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>npm install\r\n<\/code><\/pre>\n<\/div>\n<p>Now let\u2019s take a look at those important configuration files:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>cat localConfig.json \r\n<span class=\"o\">{<\/span>\r\n  <span class=\"s2\">\"qnaUri\"<\/span> : <span class=\"s2\">\"http:\/\/qnaservice.cloudapp.net\/KBService.svc\/GetAnswer?kbId=a3dae93561d24529b11a4c0eb8acd444&amp;question=\"<\/span>,\r\n  <span class=\"s2\">\"BOT_APP_ID\"<\/span> : <span class=\"s2\">\"&lt;YOUR_APP_ID_HERE&gt;\"<\/span>,\r\n  <span class=\"s2\">\"BOT_APP_PASSWORD\"<\/span> : <span class=\"s2\">\"&lt;YOUR_APP_PASSWORD_HERE&gt;\"<\/span>\r\n<span class=\"o\">}<\/span>\r\ntobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>\r\n<\/code><\/pre>\n<\/div>\n<p><strong>localConfig.json<\/strong> is where we\u2019re keeping all our secrets and configurables. You\u2019ll need to edit this to fill out the details that will be specific to your Bot. <em>It\u2019s probably not a good idea to check this file into any publicly visible source control. Doing so could allow a malicious user to impersonate your app<\/em>.<\/p>\n<p>Here\u2019s what those config settings do:<\/p>\n<p><strong>qnaUri<\/strong> &#8211; The QnA service endpoint. This one is preconfigured to point at a endpoint that serves the FAQ content for the Microsoft Bot Framework SDK. You\u2019ll be replacing this with your own in the next section.<\/p>\n<p><strong>BOT_APP_ID<\/strong> &#8211; The Microsoft Application ID. Replace with the ID you were assigned when you registered your Bot.<\/p>\n<p><strong>BOT_APP_PASSWORD<\/strong> &#8211; The application password. Replace with the one you got at the same time as the application ID.<\/p>\n<p>There\u2019s just one more secret to fill in, an attribute for the webchat control:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>cat html\/index.html \r\n&lt;html&gt;\r\n&lt;head\/&gt;\r\n&lt;body&gt;\r\n&lt;iframe <span class=\"nv\">src<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"https:\/\/webchat.botframework.com\/embed\/qnabot?s=&lt;YOUR_APP_SECRET_HERE&gt;\"<\/span> <span class=\"nv\">style<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"height: 502px; max-height: 502px;\"<\/span>&gt;&lt;\/iframe&gt;\r\n&lt;\/body&gt;\r\n&lt;\/html&gt;\r\n<\/code><\/pre>\n<\/div>\n<p><strong>YOUR_APP_SECRET_HERE<\/strong> &#8211; You\u2019ll find this on your Bot configuration page (the same place you registered your Bot). Click \u2018Edit\u2019 next to Web Chat in the Channels section and you\u2019ll be shown the secret to copy and paste into the IFRAME src attribute.<\/p>\n<h3 id=\"running-the-bot\">Running the Bot<\/h3>\n<p>With all that out of the way we can now run our Bot for the first time:<\/p>\n<div class=\"language-bash highlighter-rouge\">\n<pre class=\"highlight\"><code>tobe ~\/src\/qnamakerbot <span class=\"nv\">$ <\/span>node app.js \r\nrestify listening to http:\/\/[::]:3978\r\n<\/code><\/pre>\n<\/div>\n<p>Head over to <a href=\"http:\/\/localhost:3978\">http:\/\/localhost:3978<\/a> and you should be looking at something like this:<\/p>\n<p> <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2016\/09\/webchat.png\" alt=\"Image webchat\" width=\"601\" height=\"1005\" class=\"aligncenter size-full wp-image-11114\" srcset=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/webchat.png 601w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/webchat-179x300.png 179w\" sizes=\"(max-width: 601px) 100vw, 601px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>Awesome! We now have a fully functioning Bot. Anything you type in the web chat control is sent, with some help from ngrok, via the Bot Framework to the app running on your desktop. From there a call is made to an existing QnA service that has been preconfigured to answer questions from the Micrsoft Bot SDK FAQ. Go ahead and ask a it a few questions to see everything working.<\/p>\n<p>Now let\u2019s go right ahead and set up a new QnA Service that can answer questions from whatever FAQ-style content we have to hand.<\/p>\n<h3 id=\"configure-the-qna-service\">Configure the QnA Service<\/h3>\n<p>Head one over to the QnA Maker <a href=\"https:\/\/qnamaker.botframework.com\/\">site<\/a> (MSA login required).\nChoose Create new QnA service and give your bot a name.<\/p>\n<p>Now we get to actually provide our QnA content. We have three options here which we can mix and match to create our knowledge base.<\/p>\n<h4 id=\"web-crawling\">Web Crawling<\/h4>\n<p>The simplest is just to point QnA at an existing webpage which already contains our FAQ. QnA will crawl the site structure looking for text patterns that look like question\/answer pairs to build its knowledge base.<\/p>\n<p>The pre-configured QnA Service we\u2019ve been using so far was trained on the Microsoft Bot SDK FAQ. All I had to do was paste this uri: <a href=\"https:\/\/docs.botframework.com\/en-us\/faq\/\">https:\/\/docs.botframework.com\/en-us\/faq\/<\/a> into the configuration to have it learn the FAQ contents.<\/p>\n<h4 id=\"editorial-style\">Editorial Style<\/h4>\n<p>We can also add colon-separated QnA pairs directly into the text box in the editorial section. This is useful for adding bits of custom content that your existing FAQ base may not already contain.<\/p>\n<h4 id=\"structured-text\">Structured Text<\/h4>\n<p>You\u2019ll get the most control over exactly what content QnA will process by using the final option which is to bulk upload the question\/answer pairs in a text document structured so that the ingest process can easily separate the two. I had best success with the tab-separated option. NOTE: The .suffix of the file is significant here. <strong>If you want to use a tab-separated document it <em>must<\/em> have the .tsv suffix<\/strong> to be recognised as such. It\u2019s also important that there are no duplicate questions in our document as this will cause the parser to exit early and you\u2019ll end up with fewer questions than you thought you had.<\/p>\n<h4 id=\"extraction\">Extraction<\/h4>\n<p>Once you\u2019ve shown QnA where to get hold of your content choose \u2018Extract\u2019 to have it go and build its knowledge base. This should only take a short amount of time.<\/p>\n<p>Once that\u2019s done you\u2019ll be able to play around with the knowledge base directly. Type a few questions into the web chat. If you see a few answers that don\u2019t match your questions you now have the opportunity to tweak those.<\/p>\n<h4 id=\"training-the-knowledge-base\">Training the Knowledge Base<\/h4>\n<p>If some of the answers you\u2019re getting don\u2019t match the questions you asked QnA Maker gives you a super simple method to refine the knowledge base.<\/p>\n<p> <img decoding=\"async\" src=\"https:\/\/devblogs.microsoft.com\/cse\/wp-content\/uploads\/sites\/55\/2016\/09\/training.png\" alt=\"Image training\" width=\"800\" height=\"634\" class=\"aligncenter size-full wp-image-11113\" srcset=\"https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/training.png 800w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/training-300x238.png 300w, https:\/\/devblogs.microsoft.com\/ise\/wp-content\/uploads\/sites\/55\/2016\/09\/training-768x609.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/p>\n<p>Everytime QnA Maker returns you an answer in training mode it also show you a selection of the other high-scoring answers for that questions. If you think one of those fits the question better just click on the bubble on the left-hand side to let QnA Maker know you think that was a better response. If none seem to match choose \u2018None of the above\u2019 to tell QnA to look for other answers next time.<\/p>\n<p>Similarly if you think there are different ways of asking a question that QnA is not picking up on, you have the option to provide those in the edit box on the right hand side.<\/p>\n<p>Most of the time QnA Maker does a really good job of figuring this stuff out for itself but it\u2019s great to have the option to fine-tune the knowledge base should you need it.<\/p>\n<p>Just click \u2018Re-Train\u2019 any time you want to try the new model and you should now find your edits taken into account when you next ask those problematic questions.<\/p>\n<h4 id=\"publish-the-qna-service\">Publish the QnA Service<\/h4>\n<p>As easy as clicking the Publish button. You\u2019ll be taken to a page showing you the address of your service endpoint. Copy this and use it as the value for the \u2018qnaUri\u2019 variable in the localConfig.json at the project root.<\/p>\n<p>Restart the app and you should now be able to use the webchat control to query your FAQ content using natural language.<\/p>\n<h3 id=\"thats-it\">That\u2019s it!<\/h3>\n<p>Well.. not quite. Of course once you\u2019re ready to unleash your FAQ answering Bot on the world you\u2019ll need to deploy your Bot app in the cloud somewhere and update the Bot configuration to point at that new URI. A full guide can be found <a href=\"https:\/\/docs.botframework.com\/en-us\/node\/builder\/guides\/deploying-to-azure\/#navtitle\">here<\/a>.<\/p>\n<h3 id=\"summing-up\">Summing Up<\/h3>\n<p>QnA Maker provides us with a very low friction path to servicing FAQ-style content through a conversational interface. All the complex natural language processing tasks usually associated with projects of this kind have been taken care of for us. QnA Maker is an excellent way to dip our toe into the world of conversational interfaces and get real results quickly.<\/p>\n<p>And remember.. the code is freely available under an Open Source License from GitHub:<\/p>\n<p><a href=\"https:\/\/github.com\/CatalystCode\/qnamakerbot\/tree\/minimal\">https:\/\/github.com\/CatalystCode\/qnamakerbot\/tree\/minimal<\/a><\/p>\n<h3 id=\"suggestions-for-further-development\">Suggestions for Further Development<\/h3>\n<p>What\u2019s been presented here is close to the minimal possible solution for clarity\u2019s sake. It\u2019s easy to add whatever extra features we can imagine. Here are some ideas:<\/p>\n<h4 id=\"consider-using-metadata-to-control-response-rendering\">Consider using metadata to control response rendering<\/h4>\n<p>One technique that I\u2019ve seen used to good effect is to add metadata to the answer text which is stripped out and used to influence how the response is rendered before it\u2019s returned to the user. The Botbuilder SDK allows a variety of <a href=\"https:\/\/docs.botframework.com\/en-us\/csharp\/builder\/sdkreference\/attachments.html\">rich media responses<\/a> to be returned to the user and does a good job of translating these into forms appropriate for the channels. Where appropriate you might choose to return a HeroCard or perhaps an embdedded video instead of block of text.<\/p>\n<h4 id=\"add-telemetry-to-better-understand-how-well-the-faq-performing\">Add telemetry to better understand how well the FAQ performing<\/h4>\n<p>Data\u2019s always useful to understand systems are being used. The QnA maker returns us a confidence score along with the answer text. In situations where that confidence falls below a certain threshold we could ask the user if the previous answer was helpful. The responses to that question could help identifiy areas where the content might extra attention to better serve the customer\u2019s need.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Using Microsoft Bot Framework&#8217;s QnA Maker to create a conversational interface for publishing FAQ-style content.<\/p>\n","protected":false},"author":21384,"featured_media":11110,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[13,19],"tags":[109,110,231,250,268,301],"class_list":["post-2139","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-bots","category-machine-learning","tag-bot-framework-sdk","tag-bots","tag-language-understanding-intelligent-service-luis","tag-microsoft-cognitive-services","tag-natural-language-processing","tag-qna-maker"],"acf":[],"blog_post_summary":"<p>Using Microsoft Bot Framework&#8217;s QnA Maker to create a conversational interface for publishing FAQ-style content.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2139","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/users\/21384"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/comments?post=2139"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/posts\/2139\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media\/11110"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/media?parent=2139"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/categories?post=2139"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/ise\/wp-json\/wp\/v2\/tags?post=2139"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}