Get raw voice from speech- BotBuilder v4

Question

Working on the Bot Builder, I'm looking for a solution where I could get the realtime speech/audio of the speaker who spoke to the bot as attachment. Is it possible? Following is my code.

     public async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default(CancellationToken))
     {
            if (turnContext.Activity.Type == ActivityTypes.Message)
            {
                // Get the conversation state from the turn context.
                var state = await _accessors.CounterState.GetAsync(turnContext, () => new CounterState());

                // Bump the turn count for this conversation.
                state.TurnCount++;

                // Set the property using the accessor.
                await _accessors.CounterState.SetAsync(turnContext, state);

                // Save the new turn count into the conversation state.
                await _accessors.ConversationState.SaveChangesAsync(turnContext);

                Activity activity = new Activity();
                activity.Text = turnContext.Activity.Text;
                activity.Speak = turnContext.Activity.Speak;
                await turnContext.SendActivityAsync(activity.Text,activity.Speak,"acceptingInput",cancellationToken);
            }
            else
            {
                await turnContext.SendActivityAsync($"{turnContext.Activity.Type} event detected");
            }
    }

Please note that I am developing a bot which will receive voice ( from cortana channel ) and will convert it in .wav .

Answer 1

updated 2019-02-18

You can use platforms like botservice/cortana to do the text to speech and speech to text for you. (A Cortana skill is a voice enabled chatbot.)

Try these

• https://github.com/Microsoft/BotBuilder-Samples

• https://learn.microsoft.com/en-us/azure/bot-service/?view=azure-bot-service-4.0

• https://learn.microsoft.com/en-us/cortana/skills/overview

• https://www.youtube.com/watch?v=vCgqZPbQBQk (shameless plug)

However, what you are proposing to "get the raw voice" via these tools is not supported because of privacy issues. Internally this data is guarded, short lived and transient. Effectively, you are building your own agent to open the mic, record some audio, and then send it off for processing to speech service. (Your application would be responsible for managing and securing this data.)

• https://learn.microsoft.com/azure/cognitive-services/speech-service/rest-apis

• https://learn.microsoft.com/azure/cognitive-services/speech-service/speech-sdk