Speech Recognition in .NET MAUI with CommunityToolkit
Note: This is a guest blog post by Vladislav Antonyuk, who is a senior software engineer at DataArt and a core contributor of the .NET MAUI Community Toolkit.
The .NET MAUI Community Toolkit is a collection of extensions and components that can be used to extend the functionality of .NET MAUI apps. The toolkit is open-source and community-driven, and it is constantly being updated with new features and improvements.
One of the features that the .NET MAUI Community Toolkit offers is Speech To Text. This allows converting spoken words into text, which can be used in a variety of ways. For example, users could use speech-to-text to create a voice-activated assistant or to transcribe audio recordings.
Here’s an example of how to use SpeechToText in C#:
var isGranted = await SpeechToText.Default.RequestPermissions(cancellationToken);
if (!isGranted)
{
await Toast.Make("Permission not granted").Show(CancellationToken.None);
return;
}
var recognitionResult = await SpeechToText.Default.ListenAsync(
CultureInfo.GetCultureInfo("uk-ua"),
new Progress(partialText =>
{
RecognitionText += partialText + " ";
}), cancellationToken);
if (recognitionResult.IsSuccessful)
{
RecognitionText = recognitionResult.Text;
}
else
{
await Toast.Make(recognitionResult.Exception?.Message ?? "Unable to recognize speech").Show(CancellationToken.None);
}This code requests microphone and speech recognition permissions, then starts listening for speech input, in a result it will set any recognized text to the RecognitionText variable.
When using SpeechToText, it captures and handles all exceptions while returning the result of the operation. However, if you prefer to specifically handle certain exceptions, such as when the user cancels the operation, you can enclose your code within a try/catch block and utilize the EnsureSuccess method:
var isGranted = await SpeechToText.Default.RequestPermissions(cancellationToken);
if (!isGranted)
{
await Toast.Make("Permission not granted").Show(CancellationToken.None);
return;
}
var recognitionResult = await SpeechToText.Default.ListenAsync(
CultureInfo.GetCultureInfo("uk-ua"),
new Progress(), cancellationToken);
recognitionResult.EnsureSuccess();
await Toast.Make($"RecognizedText: {recognitionResult.Text}").Show(cancellationToken);Note:
SpeechToTextrequires additional permissions for the app.Please read the documentation to correctly set up the application.
Summary
SpeechToText is a powerful new feature that can be found as part of the CommunityToolkit.Maui library. It can be used to create a variety of more accurate, more responsive, and more engaging speech-enabled applications.
When utilizing speech-to-text, there are several additional factors to take into account:
- The availability of an Internet connection may be necessary depending on the chosen recognition language. On Windows, the speech recognition system automatically adjusts between online and offline modes based on Internet accessibility.
- The accuracy of speech-to-text is influenced by factors such as microphone quality and the surrounding environment.
- Enhancing the accuracy of speech-to-text can be achieved by using a noise-canceling microphone and speaking clearly and at a slower pace.
- Training the speech recognizer with your own voice is another method to improve the accuracy of speech-to-text.
Finally, be sure to check out the full release notes for .NET MAUI CommunityToolkit version 5.2.0 for even more great resources for .NET MAUI developers.
Light
Dark
9 comments
Is this support recognize chinese?
it supports all languages, that your device support. you may need to install additional language packs to add more languages
thank you then how to download it?
Thank you Vladislv.
I checked it out with Android Level 29.
The result is surprisingly good!
The following line of code does not seem to have any effect, however:
CultureInfo.GetCultureInfo(“uk-ua”)
I made tests with en-GB, de-DE, pl-PL, fr-FR and found out, that the system language is considered. If you have more than one language selected in your Android phone, then the second language works, too. But not the third.
When I moved Polish from 3rd position in the Android settings to the top of the selected languages, the results were very good, too!
So, this is very exiting! We can leverage speech recognition with .NET MAUI for free instead of using expensive services like Azure Cognitive Services.
Thank you for your feedback.
As for the culture parameter. the output result (text) is returned in the language you set. If you set a culture that is not supported by your device, it most likely uses the default one. You can set one language, but speak another, but the result may differ from what you expect.
Hi Vladislav, I tried your example but on Windows it always throws “A task was canceled.” Any idea?
On Android, when I added a callback to the Progress, i.e.,
, it frequently throws “Expression not supported.” Any idea?
I couldn’t compile the example at first. < string > is missing after Progress. It is correct in the documentation.
This looks very interesting. Why does it have a MAUI dependency? The MAUI community toolkit repo says that it is “for development with .NET MAUI”. Instead this should be usable with other cross-platform and single-platform approaches too. We use libraries including SkiaSharp, Xamarin.Essentials, Microsoft.Identity.Client, Serilog, and others, and they have no trouble working across platforms despite using platform-specific approaches.
This feature doesn’t depend on .NET MAUI. But the whole library does. Maybe one day it will be promoted to .NET MAUI Essentials.