LUIS & Speech API Integration

Ever wondered how to integrate speech and AI into your application? It’s easy with this tutorial from Premier Developer consultants Adel Ghabboun and Kunal Sinha!

Have you ever wondered what are some of the technologies behind personal home assistants such as Cortana, Alexa and Home? Whether you want to tell your personal AI to open Spotify and play a song for you or schedule a meeting on your calendar, Microsoft’s Cognitive Services along with LUIS (Language Understanding Intelligent Service) AI can provide you with exactly this functionality.

In this blog, we’ll be talking about how to integrate the Speech API with LUIS to construct applications that can perform tasks just by speaking to them.

For this walkthrough, we’ll be using:

The cognitive service Speech API to recognize your voice commands.
LUIS to predict the user’s intents, which will be interpreted by the application to perform specific tasks.
Microphone configuration to send data to LUIS and retrieve a response.

Steps:

To get started, get your Bing Speech API key here
Next, create an account with LUIS here. Once you have registered, log into LUIS and create a new app.
In order to train our AI to pick up on specific “intents” (what we want it to do), we will create an intent and list out a few examples of what the user might say to trigger this task. For example, if our intent is to “Enroll Student”, some example trigger utterances could be:

After listing key triggers, make sure to train your model and publish. You can get more information about training your model with intents and entities by looking at the LUIS documentation.
Once we have listed out our utterances / entities, and trained our model, we are ready to handle the intent received in our application using the Speech API. We’ll configure our Microphone to read data from the input and send it to the Speech Recognition Service. In order to link LUIS with the Speech API, add the SpeechSubscriptionId, LuisAppId and LuisSubscriptionId in your method:
```
MicrophoneRecognitionClient _client = SpeechRecognitionServiceFactory.CreateMicrophoneClientWithIntent(
     "en-us",
     recognitionConfig.SpeechSubscriptionId,
     recognitionConfig.LuisAppId,
     recognitionConfig.LuisSubscriptionId);

_client.StartMicAndRecognition();

_client.OnIntent += ((s, e) => {
     // do stuff
});
```
For more information on setting up the Speech API, look at this documentation.
Once we have obtained the intent/entities from our LUIS service, we can perform specific actions in the _client.OnIntent method i.e. with our Play Music intent, we can call the Spotify API with the entities returned in our response to make a query and retrieve the song that matches.

Summary:

As you can see, the process of setting up both LUIS and the Speech API is very straightforward and allows you to build out applications that you can directly speak to and perform tasks. For more details, see references below.