Hot word detection fails to restart when app is in background, Android?

Question

So in my assistant app, I detect when user says a specific word then I launch my app when in background. I use a library called PocketSphinx in a foreground service to detect the hot word and it works rather good. Here is the foreground service I'm launching:

public class HotKeyWordDetectionService extends Service
    implements RecognitionListener {


private static final String KWS_SEARCH = "wakeup";

/* Keyword we are looking for to activate menu */
private static final String KEYPHRASE = "hello buddy";
private SpeechRecognizer recognizer;
private ExecutorService service;



@Override
public int onStartCommand(Intent intent, int flags, int startId) {
    startForeground(1, createNotification());
    service = Executors.newFixedThreadPool(3);
    // Recognizer initialization is a time-consuming and it involves IO,
    // so we execute it in a separate thread.
    init();
    return START_NOT_STICKY;
}


public void init() {
    service.submit(() -> {
        try {
            Assets assets = new Assets(HotKeyWordDetectionService5.this);
            File assetDir = assets.syncAssets();
            setupRecognizer(assetDir);
            Log.i("sphinx_test", "setup success: ");
            handler.sendEmptyMessage(0);
        } catch (IOException e) {
            Log.i("sphinx_test", "setup failed: "+"Failed to init recognizer " + e.getMessage());
            e.printStackTrace();
        }
    });
}

private Handler handler = new Handler(message -> {
    switchSearch(KWS_SEARCH);
    return false;
});


@Override
public void onResult(Hypothesis hypothesis) {
    if (hypothesis != null) {
        String text = hypothesis.getHypstr();
        if (text.equals(KEYPHRASE)) 
            launchMyApp(); 
    }
}


private void switchSearch(String searchName) {
    recognizer.stop();
    recognizer.startListening(KWS_SEARCH);
}

private void setupRecognizer(File assetsDir) throws IOException {
    // The recognizer can be configured to perform multiple searches
    // of different kind and switch between them

    recognizer = SpeechRecognizerSetup.defaultSetup()
            .setAcousticModel(new File(assetsDir, "en-us-ptm"))
            .setDictionary(new File(assetsDir, "cmudict-en-us.dict"))
            .setRawLogDir(assetsDir) 
            .getRecognizer();
    recognizer.addListener(this);

    // Create keyword-activation search.
    recognizer.addKeyphraseSearch(KWS_SEARCH, KEYPHRASE);

    // Create grammar-based search for selection between demos
    File menuGrammar = new File(assetsDir, "menu.gram");
    recognizer.addGrammarSearch(MENU_SEARCH, menuGrammar);

    // Create grammar-based search for digit recognition
    File digitsGrammar = new File(assetsDir, "digits.gram");
    recognizer.addGrammarSearch(DIGITS_SEARCH, digitsGrammar);

    // Create language model search
    File languageModel = new File(assetsDir, "weather.dmp");
    recognizer.addNgramSearch(FORECAST_SEARCH, languageModel);

    // Phonetic search
    File phoneticModel = new File(assetsDir, "en-phone.dmp");
    recognizer.addAllphoneSearch(PHONE_SEARCH, phoneticModel);
}

}

Problem rises when user uses another app to record audio like WhatsApp. This causes my app to lose control of microphone. After user is finished with microphone in the other app, my app won't be able to regain the control over microphone, at least in Android version 11 and above, unless user launches my app again. Here is the error log when my foreground service tries to resume the hot word detection, but it fails:

>com.parts.assitant E/IAudioFlinger: createRecord returned error -1
>com.parts.assitant E/AudioRecord: createRecord_l(5263): AudioFlinger could not create record track, status: -1
>com.parts.assitant W/AudioRecord: restoreRecord_l(5263): failed status -1, retries 2
>com.parts.assitant I/BpBinder: onLastStrongRef automatically unlinking death recipients: <uncached descriptor>

Point is I launch the foreground service when the app is in foreground, so I believe it is not included in new limits introduced in Android 11 and above. But still my process cannot regain the control over microphone until the app is relaunched (is again in the foreground). How can I solve the issue?

Answer 1

After some more research I found out the solution. In Android 11 and above I have to add android:foregroundServiceType="microphone" to my foreground service declaration in AndroidManifest.xml:

<service android:name=".HotKeyWordDetectionService" android:foregroundServiceType="microphone"/>

And also to my service in onStartCommand():

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R)
    startForeground(1, createNotification(), ServiceInfo.FOREGROUND_SERVICE_TYPE_MICROPHONE);
else
    startForeground(1, createNotification());

Now when I use WhatsApp or any other app to record audio, my service stops working and when I'm done with microphone, my service resumes the successful detection process.