Azure Speech Services start_continuous_recognition() is not letting streamlit st.write() the recognized text

Question

I am trying to build a Streamlit application using Azure Speech Services. I would like to show the Live Recognized Content on the UI.

If I am using recognize_once() logic mentioned below, I am able to see the output in the app UI, whereas when I am trying to use start_continuous_recognition(), somehow the st.write() logic is being skipped.

Recognizing Once Logic:

def speech_recognize_once_from_mic():
    # Set up the speech config and audio config
    speech_config = speechsdk.SpeechConfig(subscription=subscription_key, region=service_region)

    audio_config = speechsdk.AudioConfig(use_default_microphone=True)

    # Create a speech recognizer with the given settings
    speech_recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)

    st.write("Speak into your microphone.")
    result = speech_recognizer.recognize_once_async().get()

    # Check the result
    if result.reason == speechsdk.ResultReason.RecognizedSpeech:
        return f"Recognized: {result.text}"
    elif result.reason == speechsdk.ResultReason.NoMatch:
        return "No speech could be recognized"
    elif result.reason == speechsdk.ResultReason.Canceled:
        cancellation_details = result.cancellation_details
        return f"Speech Recognition canceled: {cancellation_details.reason}"
    else:
        return "Unknown error"

# Simple UI for processing the audio input

st.title("Azure Speech Service with Streamlit")

if st.button('Start speech recognition'):
    recognition_result = speech_recognize_once_from_mic()
    st.write(recognition_result)

Continuous Recognition Logic:

def recognized_callback(evt):
    # Callback function that appends recognized speech to chunks
    global chunks
    if evt.result.reason == speechsdk.ResultReason.RecognizedSpeech:
        chunks.append(evt.result.text)
        print(f"Recognized: {evt.result.text}")
        st.write(chunks)
        print('Done')


def process_audio():
    # Set up the recognizer
    recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)
    recognizer.recognized.connect(recognized_callback)
    recognizer.session_stopped.connect(session_stopped_callback)

    # Store the recognizer in the session state to access it later
    st.session_state.recognizer = recognizer

    # Start continuous recognition
    recognizer.start_continuous_recognition()

How can I fix this?

Thanks in advance! :)

Answer 1

I tried the following code with st.write() in a Streamlit application using Azure Speech Services and received the Live Recognized text on the browser UI.

Code :

import streamlit as st
import azure.cognitiveservices.speech as speechsdk

chunks = []

subscription_key = "<speech_key>"
service_region = "<speech_region>"
speech_config = speechsdk.SpeechConfig(subscription=subscription_key, region=service_region)
audio_config = speechsdk.AudioConfig(use_default_microphone=True)

def recognized_callback(evt):
    global chunks
    if evt.result.reason == speechsdk.ResultReason.RecognizedSpeech:
        chunks.append(evt.result.text)
        print(f"Recognized: {evt.result.text}")

def main():
    st.title("Azure Speech Service with Streamlit")
    
    recognizer = speechsdk.SpeechRecognizer(speech_config=speech_config, audio_config=audio_config)
    recognizer.recognized.connect(recognized_callback)
    recognizer.start_continuous_recognition()
    
    st.write("Speak into your microphone.")
    while True:
        if chunks:
            st.write(chunks)
            chunks.clear()
            st.write("Done")

if __name__ == "__main__":
    main()

Browser Output :

I got the following output for Streamlit:

I spoke the following sentences, and it successfully converted speech to text as shown below:

Output :

The program ran successfully, and I received the recognition text output in the VS Code terminal as shown below.

C:\Users\xxxxxxxxx\Documents\xxxxxxxx>streamlit run app.py

  You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501
  Network URL: http://192.168.0.126:8501

Recognized: Hi, Kamali.
Recognized: Welcome.
Recognized: Welcome to my world, Kamali.
Recognized: How are you?
Recognized: Very good to see you.
Recognized: I am very happy for you.
Recognized: Let's continue the recognition.
Recognized: Hi, Priya.
Recognized: Welcome.
Recognized: I am very happy to see you.