Android media codec: Mismatch between encoder and decoder buffer sizes

Question

I'm trying to implement a transcoding pipeline in Android using mediacodec library in cpp. I have a h265 decoder that is running in surface mode. I changed this to buffer mode by removing the surface argument while configuring the decoder. Now I am able to see that the output buffer has some data after decoding. My plan is to use the output buffer and copy it into the input of a h264 encoder.

The issue I am seeing is that the size of output buffer has of decoder is larger than the size of the input buffer of the encoder. I have checked that the color format is the same between the two. I'm not sure why there's a difference. For a 1080p video in YUV420 the size of the buffer should be

Y Plane Size: 2,073,600 bytes
U Plane Size: 518,400 bytes
V Plane Size: 518,400 bytes
Total Frame Size: 2,073,600 + 518,400 + 518,400 = 3,110,400 bytes

Code snippet for configuring encoder:

AMediaFormat* avc_format = AMediaFormat_new();
AMediaFormat_setString(avc_format, AMEDIAFORMAT_KEY_MIME, "video/avc");
AMediaFormat_setInt32(avc_format, AMEDIAFORMAT_KEY_WIDTH, 1920);
AMediaFormat_setInt32(avc_format, AMEDIAFORMAT_KEY_HEIGHT, 1080);
AMediaFormat_setInt32(avc_format, AMEDIAFORMAT_KEY_BIT_RATE, 17000000);
AMediaFormat_setInt32(avc_format, AMEDIAFORMAT_KEY_I_FRAME_INTERVAL, 1);
AMediaFormat_setFloat(avc_format, AMEDIAFORMAT_KEY_FRAME_RATE, 30.0);
AMediaFormat_setInt32(avc_format, AMEDIAFORMAT_KEY_COLOR_FORMAT, 21);
avc_encoder_ = AMediaCodec_createCodecByName("OMX.qcom.video.encoder.avc");
AMediaCodec_configure(
        avc_encoder_, avc_format, NULL, NULL, AMEDIACODEC_CONFIGURE_FLAG_ENCODE);

Code snippet for getting the buffers:

ssize_t codec_out_buf_idx = AMediaCodec_dequeueOutputBuffer(hevc_decoder_ &info, timeout);
ssize_t bufidx = AMediaCodec_dequeueInputBuffer(avc_encoder_, timeout);
if (bufidx >= 0) {
    uint8_t* buf = AMediaCodec_getInputBuffer(avc_encoder_, bufidx, &bufsize);
    size_t codec_out_bufsize = 0;
    uint8_t* codec_out_buf = AMediaCodec_getOutputBuffer(hevc_decoder_, codec_out_buf_idx, &codec_out_bufsize);
    if (codec_out_bufsize > bufsize) {
       return;
    }
 }

The issue I am seeing is that the size of output buffer has of decoder is larger than the size of the input buffer of the encoder. I have checked that the color format is the same between the two. I'm not sure why there's a difference. But I see that the decoder Output buffer size 3133440 is larger than input buffer size 3110400. Also on the output format changed callback for the decoder I get this:

Output format changed
Width: 1920, Height: 1080
Frame Rate: 30
Color Format: 21
MIME Type: video/raw

I tried different color formats but I checked that the color format matches between the encoder and decoder. Any ideas why the buffer sizes differ? Is there a better way of doing this?

Answer 1

Some decoders prefer to work with buffers having dimensions that are multiples of 16 (possibly because they work with 16 x 16 pixel blocks).

So in your case the decoder allocates buffers with dimensions 1920 x 1088 (1080 is not a multiple of 16) resulting in the buffer size of 3,133,440 bytes. And the padding is supposed to be ignored.

When the decoder is configured in the "surface mode" this is not a problem.

However in the "buffer mode" the data from decoder's buffers have to be copied to encoder's buffers. But the encoder allocates buffers based on the given dimensions of the video, resulting in smaller buffers.

One way to overcome this problem is to copy the data row by row taking into account the given crop rectangle.