reduce encoder frame rate

Question

I am tying to limit the frame rate to 5-10 fps. The encoded frame are being sent though web-socket connection and my goal is to limit bandwidth to 1Mbps while having good quality for larger resolution.

my current attempt is:

    val format =
            MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_VP8, size.width, size.height)
        format.setInteger(MediaFormat.KEY_COLOR_FORMAT,
            MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface);
        format.setInteger(MediaFormat.KEY_BIT_RATE, 1000000);
        format.setInteger(MediaFormat.KEY_BITRATE_MODE,  MediaCodecInfo.EncoderCapabilities.BITRATE_MODE_CBR_FD);
        format.setInteger(MediaFormat.KEY_FRAME_RATE, 1);
        format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
        format.setInteger(MediaFormat.KEY_MAX_FPS_TO_ENCODER, 1);
encoderInput = mEncoder!!.createInputSurface()
encoderInput.setFrameRate(1f,Surface.FRAME_RATE_COMPATIBILITY_FIXED_SOURCE,Surface.CHANGE_FRAME_RATE_ALWAYS)

I tired both setting the frame rate in the format and on the surface but it is still not limiting it. based on this discussion MediaCodec KEY_FRAME_RATE seems to be ignored

The bandwidth does follow the bit rate set in the format but not he frame rate so the quality is bad.

so how to reduce the frame rate ? preferably in a way that support older version of android.

Answer 1

The frame-rate value that you use to configure the encoder doesn't change the video stream. It is mainly a hint for the player / decoder about what frame-rate to expect.

In order to actually change the frame-rate of the video stream somebody must do a process called sample-rate conversion.

If you control the frames that are sent to the encoder, then one way to reduce the video frame-rate is to drop every Nth frame. But this will likely result in aliasing. So, for good quality results you also need to implement a low-pass filter.

If the frames come from a third-party library, then look in the documentation for a way to request a different frame-rate. Here's an example of doing so using the Camera2 API.