Azure Cognitive Services offers a wide range of AI-powered services that can be utilized to enhance applications and services. In this article and associated demo project, I’m using Azure SDK client libraries to transcribe and translate audio files. By applying Azure Cognitive Services, you can easily convert audio files in one language to text in another language.<\/p>\n
Before diving into the tutorial, ensure that you have the following prerequisites:<\/p>\n
python-dotenv<\/code> package installed. You can install them using the following command:<\/li>\n<\/ol>\n pip install azure-cognitiveservices-speech azure-ai-translation-text python-dotenv<\/code><\/pre>\n\n- Get the demo project with the full script. You can clone the repo to your local machine using git:\n
git clone https:\/\/github.com\/mario-guerra\/azure-speech-translator.git<\/code><\/pre>\n<\/li>\n<\/ol>\nSet up the environment<\/h3>\n
To set up your environment, retrieve the keys and endpoints for both the Speech Service and Translator Service from your Azure account.<\/p>\n
To locate the keys and endpoints for both the Speech Service and Translator Service in your Azure account, follow these steps:<\/p>\n
\n- Sign in to the Azure Portal<\/strong>: Visit the Azure portal<\/a> and sign in with your Azure account credentials.<\/li>\n
- Access the Speech Service<\/strong>:\n
\n- In the left-hand menu, select “All services.”<\/li>\n
- In the search box, type “Speech” and select “Speech” from the results.<\/li>\n
- Choose the Speech Service you have created or create a new one if you haven’t already.<\/li>\n<\/ul>\n<\/li>\n
- Retrieve the Speech Service key and endpoint<\/strong>:\n
\n- Once you have accessed your Speech Service, navigate to the “Keys and Endpoint” on the left-side menu under the “Resource Management” section.<\/li>\n
- Copy the
Key1<\/code> or Key2<\/code> value (both are valid) and “Location\/Region” value. These values are used as your AZURE_SPEECH_KEY<\/code> and AZURE_SERVICE_REGION<\/code>, respectively.<\/li>\n<\/ul>\n<\/li>\n- Access the Translator Service<\/strong>:\n
\n- Go back to the “All services” menu and search for “Translator.”<\/li>\n
- Select “Translator” from the results.<\/li>\n
- Choose the Translator Service you have created or create a new one if you haven’t already.<\/li>\n<\/ul>\n<\/li>\n
- Retrieve the Translator Service key and endpoint<\/strong>:\n
\n- Once you have accessed your Translator Service, navigate to “Keys and Endpoint” under the “Resource Management” section in the left-hand menu.<\/li>\n
- Copy the
Key1<\/code> or Key2<\/code> value (both are valid) and the “Text Translation” endpoint value under the “Web API” tab. These values are used as your AZURE_TRANSLATOR_KEY<\/code> and AZURE_TRANSLATOR_ENDPOINT<\/code>, respectively.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\nOnce you have your keys, region, and endpoint, create a .env<\/code> file in the same directory as your script and add the following environment variables:<\/p>\nAZURE_SPEECH_KEY=<your_speech_service_key>\r\nAZURE_SERVICE_REGION=<your_speech_service_region>\r\nAZURE_TRANSLATOR_KEY=<your_translator_service_key>\r\nAZURE_TRANSLATOR_ENDPOINT=<your_translator_service_endpoint><\/code><\/pre>\nReplace the placeholder values with the appropriate keys and endpoints from your Azure account.<\/p>\n
Audio translation script overview<\/h3>\n
The demo project features an audio translation script that processes input audio files in WAV format. It transcribes the audio using Azure Speech Service and translates the resulting text into the desired target language with Azure Translator Service.<\/p>\n
This powerful tool can be invaluable for various applications, such as language learning, content localization, and accessibility services.<\/p>\n
Customize the audio translation script<\/h3>\n
The provided audio translation script can be customized to suit specific requirements. For example, you can modify the script to:<\/p>\n
\n- Add support for more languages by updating the
language_codes<\/code><\/a> and translator_language_codes<\/code><\/a> dictionaries.<\/li>\n- Adjust the timeout settings for speech recognition by modifying the values<\/a> of the
SpeechServiceConnection_InitialSilenceTimeoutMs<\/code>, SpeechServiceConnection_EndSilenceTimeoutMs<\/code>, and Speech_SegmentationSilenceTimeoutMs<\/code> properties.<\/li>\n- Implement extra error handling and logging to improve the script’s robustness and maintainability.<\/li>\n<\/ul>\n
Transcribe with continuous recognition vs. one-shot recognition<\/h3>\n
In speech recognition, two main approaches can be used: continuous recognition and one-shot recognition. Each has its own use cases and benefits. In my demo audio translation script, I chose continuous recognition for real-time translation, better handling of pauses, and greater flexibility.<\/p>\n
Continuous recognition<\/h4>\n
Continuous recognition is a speech recognition approach that processes audio input in real-time and continuously recognizes speech as it is spoken. This method is useful when dealing with long audio files or live audio streams, as it provides real-time feedback and can handle pauses or interruptions in speech.<\/p>\n
In continuous recognition, the speech recognizer listens for speech and generates results as it recognizes words and phrases. It can also raise events when it recognizes speech, allowing you to perform actions, such as translating the recognized text, as demonstrated in our script.<\/p>\n
Here’s how I set up continuous recognition using the Azure Cognitive Services Speech library in my script:<\/p>\n
\n- Configure the Speech Service by creating a
SpeechConfig<\/code> object and setting the speech recognition language and other properties:\nspeech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)\r\nspeech_config.speech_recognition_language = speech_recognition_language\r\nspeech_config.set_property(speechsdk.PropertyId.SpeechServiceConnection_InitialSilenceTimeoutMs, \"15000\")\r\nspeech_config.set_property(speechsdk.PropertyId.SpeechServiceConnection_EndSilenceTimeoutMs, \"10000\")\r\nspeech_config.set_property(speechsdk.PropertyId.Speech_SegmentationSilenceTimeoutMs, \"5000\")<\/code><\/pre>\n<\/li>\n- Define event handlers for the
recognized<\/code> and session_stopped<\/code> events:\ndef on_recognized(recognition_args, in_lang, out_lang):\r\n source_text = recognition_args.result.text\r\n print(f\"Transcribed text: {source_text}\")\r\n\r\n # Write the transcribed text to the transcription output file if specified\r\n if cmd_line_args.transcription:\r\n with open(cmd_line_args.transcription, 'a', encoding='utf-8') as f:\r\n f.write(f\"{source_text}\\n\")\r\n\r\n # Translate the transcribed text using the Azure Translator SDK\r\n try:\r\n source_language = translator_language_codes[in_lang]\r\n # Translator service supports translation to multiple languages in one pass,\r\n # so it expects a bracketed list even when translating to only one language.\r\n target_languages = [translator_language_codes[out_lang]]\r\n input_text_elements = [ InputTextItem(text = source_text) ]\r\n response = text_translator.translate(content = input_text_elements, to = target_languages, from_parameter = source_language)\r\n translation = response[0] if response else None\r\n\r\n if translation:\r\n for translated_text in translation.translations:\r\n print(f\"Translated text: {translated_text.text}\")\r\n # Write the translated text to the output file\r\n with open(cmd_line_args.output_file, 'a', encoding='utf-8') as f:\r\n f.write(f\"{translation}\\n\")\r\n\r\n except HttpResponseError as exception:\r\n print(f\"Error Code: {exception.error.code}\")\r\n print(f\"Message: {exception.error.message}\")\r\n\r\ndef on_session_stopped(args):\r\n print(\"Continuous recognition session stopped.\")\r\n global session_stopped\r\n session_stopped = True<\/code><\/pre>\n<\/li>\n- Create a
SpeechRecognizer<\/code> object using the SpeechConfig<\/code> object and an AudioConfig<\/code> object that specifies the input audio file:\naudio_input = speechsdk.audio.AudioConfig(filename=input_audio_file)\r\nspeech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_input)<\/code><\/pre>\n<\/li>\n- Connect the event handlers to the corresponding events of the
SpeechRecognizer<\/code> object and start the continuous recognition process asynchronously:\nspeech_recognizer.recognized.connect(lambda recognition_args: on_recognized(recognition_args, cmd_line_args.in_lang, cmd_line_args.out_lang))\r\nspeech_recognizer.session_stopped.connect(on_session_stopped)\r\nspeech_recognizer.start_continuous_recognition_async().get()<\/code><\/pre>\n<\/li>\n- Wait for the
session_stopped<\/code> event to be triggered before proceeding to the next audio file or terminating the script:\nwhile not session_stopped:\r\n time.sleep(0.5)<\/code><\/pre>\n<\/li>\n<\/ol>\nBy following these steps, I’ve set up continuous recognition using the Azure Cognitive Services Speech library. This approach allows the script to process audio input in real-time, handle pauses or interruptions in speech, and perform actions, such as translation, as soon as speech is recognized.<\/p>\n
One-shot recognition<\/h4>\n
One-shot recognition, also known as single-utterance recognition, processes an entire audio file or a single utterance and returns the recognition result once the audio input is complete. This approach is suitable for short audio clips or situations where real-time feedback isn’t necessary.<\/p>\n
To perform one-shot recognition, you would create a SpeechRecognizer<\/code> object, just like in continuous recognition, and then call the recognize_once_async()<\/code> method:<\/p>\nresult = speech_recognizer.recognize_once_async().get()<\/code><\/pre>\nThe recognized text can be accessed using the result.text<\/code> property. One-shot recognition is easier to implement, as it requires only a single function call, but it lacks the real-time feedback and flexibility of continuous recognition.<\/p>\nTranslate the transcriptions<\/h3>\n
Once the audio files are transcribed, the script translates the transcriptions into the desired output language using the Azure Translator library. In the on_recognized<\/code> event handler, the translation process is performed as follows:<\/p>\n\n- Retrieve the source and target language codes from the
translator_language_codes<\/code> dictionary:\nsource_language = translator_language_codes[in_lang]\r\ntarget_languages = [translator_language_codes[out_lang]]<\/code><\/pre>\n<\/li>\n- Create a list of
InputTextItem<\/code> objects containing the transcribed text:\ninput_text_elements = [ InputTextItem(text = source_text) ]<\/code><\/pre>\n<\/li>\n- Call the
translate<\/code> method of the TextTranslationClient<\/code> object, passing the input text elements, target languages, and source language:\nresponse = text_translator.translate(content = input_text_elements, to = target_languages, from_parameter = source_language)<\/code><\/pre>\n<\/li>\n- Process the translation response and write the translated text to the output file:\n
translation = response[0] if response else None\r\n\r\nif translation:\r\n for translated_text in translation.translations:\r\n print(f\"Translated text: {translated_text.text}\")\r\n # Write the translated text to the output file\r\n with open(cmd_line_args.output_file, 'a', encoding='utf-8') as f:\r\n f.write(f\"{translation}\\n\")<\/code><\/pre>\n<\/li>\n<\/ol>\nRun the audio translation script<\/h3>\n
To run the script, use the following command:<\/p>\n
python audio_translation.py --in-lang <input_language> --out-lang <output_language> <input_audio_pattern> <output_file> [--transcription <transcription_output_file>]<\/code><\/pre>\nReplace the placeholders with the appropriate values:<\/p>\n
\n<input_language><\/code>: The input language (currently supported: english, spanish, estonian, french, italian, german<\/code>)<\/li>\n<output_language><\/code>: The output language (currently supported: english, spanish, estonian, french, italian, german<\/code>)<\/li>\n<input_audio_pattern><\/code>: The path to the input audio files with a wildcard pattern (for example, .\/*.wav)<\/li>\n<output_file><\/code>: The path to the output file containing the translations<\/li>\n<transcription_output_file><\/code> (optional): The path to the output file containing the transcriptions<\/li>\n<\/ul>\nFor example:<\/p>\n
python audio_translation.py --in-lang english --out-lang spanish .\/input_audio\/*.wav output.txt --transcription transcription.txt<\/code><\/pre>\nThis command transcribes and translates all .wav<\/code> files in the input_audio<\/code> directory from English to Spanish. The translations are saved in output.txt<\/code>, and the transcriptions are saved in transcription.txt<\/code>.<\/p>\nOutput:<\/p>\n
python .\\azure_translator.py --in-lang spanish --out-lang english '.\\Spanish test.wav' .\\translation.txt\r\nProcessing audio file: .\\Spanish test.wav\r\nTranscribed text: Esta es una prueba del sistema de transmisi\u00f3n de emergencia. Solo es una prueba si esto fuera una emergencia real, estar\u00eda corriendo para salvar mi vida.\r\nTranslated text: This is a test of the emergency transmission system. It's just a test if this was a real emergency, I would be running for my life.\r\nContinuous speech recognition session stopped.<\/code><\/pre>\nConclusion<\/h3>\n
By following this tutorial and using the provided audio translation script, you can efficiently transcribe and translate audio files into different languages. This powerful tool, powered by Azure Cognitive Services, opens up numerous possibilities for language learning, content localization, and accessibility services. With the added customization options and a choice between continuous recognition and one-shot recognition, the script becomes an even more versatile solution for your audio translation needs. I encourage you to explore further and experiment with the script to discover its full potential and adapt it to various use cases.<\/p>\n
Happy coding!<\/p>\n","protected":false},"excerpt":{"rendered":"
Discover the power of Azure Cognitive Services libraries for audio transcription and translation, enhancing multilingual experiences.<\/p>\n","protected":false},"author":63526,"featured_media":2806,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1],"tags":[915,918,914,917,919,916],"class_list":["post-2803","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure-sdk","tag-cognitive-services","tag-speech-transcription","tag-speech-translation","tag-text-transcription","tag-transcribing-text","tag-translation"],"acf":[],"blog_post_summary":"
Discover the power of Azure Cognitive Services libraries for audio transcription and translation, enhancing multilingual experiences.<\/p>\n","_links":{"self":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts\/2803","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/users\/63526"}],"replies":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/comments?post=2803"}],"version-history":[{"count":0,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/posts\/2803\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/media\/2806"}],"wp:attachment":[{"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/media?parent=2803"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/categories?post=2803"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/devblogs.microsoft.com\/azure-sdk\/wp-json\/wp\/v2\/tags?post=2803"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}