android pause/resume video decoder implemented by MediaCodec

Question

I use below code to init a MediaCodec to decode raw H264 stream, which is streamed from a remote server.

    MediaFormat format = MediaFormat.createVideoFormat("video/avc", width, height);
    format.setByteBuffer("csd-0", csd0);
    format.setByteBuffer("csd-1", csd1);
    try {
        mCodec = MediaCodec.createDecoderByType("video/avc");
    } catch (IOException e) {
        throw new RuntimeException("Failed to create codec", e);
    }
    mCodec.configure(format, surface, null, 0);
    mCodec.start();

And use below code to decode video byte buffer:

      int index = mCodec.dequeueInputBuffer(-1);
        if (index >= 0) {
            ByteBuffer buffer;

            if (Build.VERSION.SDK_INT < Build.VERSION_CODES.LOLLIPOP) {
                buffer = mCodec.getInputBuffers()[index];
                buffer.clear();
            } else {
                buffer = mCodec.getInputBuffer(index);
            }
            if (buffer != null) {
                buffer.put(data, offset, size);
                mCodec.queueInputBuffer(index, 0, size, 0, flags);
            }
        }

I need to add pause resume when screen turn off. I got a exception when turn off screen:

02-12 10:40:59.249 17398 17450 E AndroidRuntime: java.lang.IllegalStateException
02-12 10:40:59.249 17398 17450 E AndroidRuntime:    at android.media.MediaCodec.native_dequeueInputBuffer(Native Method)
02-12 10:40:59.249 17398 17450 E AndroidRuntime:    at android.media.MediaCodec.dequeueInputBuffer(MediaCodec.java:2726)

What's proper way to pause/resume MediaCodec?

Answer 1

The problem is likely not in the MediaCodec but in the Surface that you used to configure it with. A Surface is an interface for a producer (e.g. MediaCodec) to exchange buffers with a consumer.

If the Surface comes from a SurfaceView or TextureView then you should be aware of its lifecycle. The Surface is created and destroyed as the view's window is shown and hidden.

After the Surface has been destroyed the buffers will not be available anymore. You should implement the relevant callbacks (e.g. surfaceCreated, surfaceDestroyed) for managing the workflow.

One option is to release the codec when the surface is destroyed and create a new one when the surface is created. Or you could reuse the same codec instance by stopping it when the surface is destroyed and configuring it using a new surface when the latter is created.

In both cases don't forget to make sure that the first buffer that you feed into the decoder is an I-frame.

More about MediaCodec states.