How can I use Google Cloud Speech-to-Text from Ionic

Question

I would like to implement the Client Library Google Cloud but there is no Ionic implementation.

How can I implement the Client Library on Ionic for using the Google Speech To Text?

For the moment, I tried to implement the NodeJS client Library but there are some problems because some people from forums say that we have to use the File native implementation from Ionic. Using File produce errors for me.

import {Injectable} from '@angular/core';
import {GoogleSpeechToText} from '@google-cloud/speech';
import {Observable} from 'rxjs';
import {Idea} from './idea.service';
import {File} from '@ionic-native/file/ngx';


@Injectable({
    providedIn: 'root'
})
export class GoogleSpeechToTextService {

    private ideas: Observable<Idea[]>;
    constructor(private GoogleSpeech: GoogleSpeechToText, private MyFile: File) {
    }

    async SpeechClient() {
        // @ts-ignore
        const client = new this.SpeechClient();
        const filename = './resources/audio/audio.raw';

        const file = File.readFileSync(filename);

        const audioBytes = file.toString('base64');
        const audio = {
            content: audioBytes
        };

        const config = {
            encoding: 'LINEAR16',
            sampleRateHertz: 1600,
            languageCode: 'en-US'
        };

        // @ts-ignore
        // @ts-ignore
        const request = {
            audio,
          config,
        };
        const [response] = await client.recognize(request);
        const transcription = response.results.map(result => result.alternative[0].transcript).join('\n');

    }
}

ERROR in ./node_modules/google-gax/build/src/operationsClient.js
Module not found: Error: Can't resolve './operations_client_config' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\google-gax\build\src'
ERROR in ./node_modules/google-auth-library/build/src/auth/googleauth.js
Module not found: Error: Can't resolve 'child_process' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\google-auth-library\build\src\auth'
ERROR in ./node_modules/google-auth-library/build/src/auth/googleauth.js
Module not found: Error: Can't resolve 'fs' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\google-auth-library\build\src\auth'
ERROR in ./node_modules/google-p12-pem/build/src/index.js
Module not found: Error: Can't resolve 'fs' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\google-p12-pem\build\src'
ERROR in ./node_modules/gtoken/build/src/index.js
Module not found: Error: Can't resolve 'fs' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\gtoken\build\src'
ERROR in ./node_modules/request/lib/har.js
Module not found: Error: Can't resolve 'fs' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\request\lib'
ERROR in ./node_modules/forever-agent/index.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\forever-agent'
ERROR in ./node_modules/gaxios/node_modules/https-proxy-agent/index.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\gaxios\node_modules\https-proxy-agent'
ERROR in ./node_modules/http-proxy-agent/index.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\http-proxy-agent'
ERROR in ./node_modules/teeny-request/node_modules/https-proxy-agent/index.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\teeny-request\node_modules\https-proxy-agent'
ERROR in ./node_modules/tough-cookie/lib/cookie.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\tough-cookie\lib'
ERROR in ./node_modules/tunnel-agent/index.js
Module not found: Error: Can't resolve 'net' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\tunnel-agent'
ERROR in ./node_modules/forever-agent/index.js
Module not found: Error: Can't resolve 'tls' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\forever-agent'
ERROR in ./node_modules/gaxios/node_modules/https-proxy-agent/index.js
Module not found: Error: Can't resolve 'tls' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\gaxios\node_modules\https-proxy-agent'
ERROR in ./node_modules/http-proxy-agent/index.js
Module not found: Error: Can't resolve 'tls' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\http-proxy-agent'
ERROR in ./node_modules/teeny-request/node_modules/https-proxy-agent/index.js
Module not found: Error: Can't resolve 'tls' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\teeny-request\node_modules\https-proxy-agent'
ERROR in ./node_modules/tunnel-agent/index.js
Module not found: Error: Can't resolve 'tls' in 'C:\Users\gdemay\Documents\Mobile-Hybrid2\node_modules\tunnel-agent'
i ｢wdm｣: Failed to compile.

    ERROR in src/app/services/google-speech-to-text.service.ts(22,27): error TS2339: Property 'readFileSync' does not exist on type 'typeof File'.

Answer 1

My understanding is that you can't unfortunately. The speech to text API in NodeJS uses backend technology.

You need to use Obvi to create a wrapper over the API.

I decided to play with this further and I managed to get it working by creating a web sockets server, then steaming audio from AudioContext to the web sockets server, then using the Google Cloud Speech to Text API client library I was able to stream audio to google and interim responses back to the browser.

Answer 2

The solution which partly worked for me was going to /node_modules/google_gax/build/src/operationsClient.js and inside, I renamed

const configData = require('./operations_client_config');

to

const configData = require('./operations_client_config.json');

Hope it helps someone.

EDIT:

if someone uses webpack and facing the problem go to webpack.config.js and add:

resolve: { extensions: [ '.json' ] }