Convert a YUVJ422 and YUVJ420 frame into YV12 in C++ with FFmpeg

40 Views Asked by At

I am currently writing a program, to convert YUVJ422 and YUVJ420 frames into YV12. The YV12 frame is then uploaded to the GPU and there converted into RGB again, but that part should work fine (more on that later).

1080p and 720p are working very good and performant (no delays or anything) already with my program, but 540p has a weird artifact in the bottom of the frame (8 pixels are green ish, but also kind of transparent, so copying the information about the brightness worked but the U and/or? V plane seem to be missing something at the end.

My thoughts are, that maybe because 540 is not even when dividing with 8, the copy operation misses somehting? Also could be some padding that is not considered? So I tried to hard-code the height to 544, before decoding and providing a height of 544 to the FFmpeg decoder, but that didn't work either and resulted in a similar output.

Another reason for the green line could be, that the shader does not take any padding into account, the height provided into the shader is 540, but I am not quite sure, if the shader is the problem, as it works for the other formats and a green line seems to indicate more, that not enough data was copied, as green lines usually mean zeroed memory, as a zero would translate to green in YUV.

I am now out of ideas, why the code fails for 540p formats, so I hope that someone already had this issue before maybe and provide some clarification, here is my code to convert the pixel data, keep in mind that the code is not fully optimized yet and I already planned to write shaders to convert from the YUVJ420 and YUVJ422 formats directly into RGBA as that would be much faster, but for now I have to take this "workaround" to convert the data first to YV12 for other reasons.

        if (mCurrentFrame->format == AV_PIX_FMT_YUVJ420P)
        {
            if (540 == mCurrentFrame->height)
            {
                int uvHeight = mCurrentFrame->height / 2;
                int offset   = 0;

                // Copy Y plane
                for (int y = 0; y < mCurrentFrame->height; ++y)
                {
                    memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[0] + mCurrentFrame->linesize[0] * y, mCurrentFrame->width);
                    offset += mCurrentFrame->width;
                }

                // Copy V plane
                for (int v = 0; v < uvHeight; ++v)
                {
                    memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[2] + mCurrentFrame->linesize[2] * v, mCurrentFrame->width / 2);
                    offset += mCurrentFrame->width / 2;
                }

                // Copy U plane
                for (int u = 0; u < uvHeight; ++u)
                {
                    memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[1] + mCurrentFrame->linesize[1] * u, mCurrentFrame->width / 2);
                    offset += mCurrentFrame->width / 2;
                }
            }
            else
            {
                int ySize  = mCurrentFrame->width * mCurrentFrame->height;
                int uvSize = (mCurrentFrame->width / 2) * (mCurrentFrame->height / 2);

                // Copy Y plane
                memcpy(decodedFrame->GetData(), mCurrentFrame->data[0], ySize);

                // Copy V plane
                memcpy(decodedFrame->GetData() + ySize, mCurrentFrame->data[2], uvSize);

                // Copy U plane
                memcpy(decodedFrame->GetData() + ySize + uvSize, mCurrentFrame->data[1], uvSize);
            }
        }
        else if (mCurrentFrame->format == AV_PIX_FMT_YUVJ422P)
        {
            int offset = 0;

            if (540 == mCurrentFrame->height)
            {
                // Copy Y plane, but linewise
                for (int y = 0; y < mCurrentFrame->height; ++y)
                {
                    memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[0] + mCurrentFrame->linesize[0] * y, mCurrentFrame->width);
                    offset += mCurrentFrame->width;
                }
            }
            else
            {
                int ySize = mCurrentFrame->width * mCurrentFrame->height;
                offset    = ySize;

                // Copy Y plane
                memcpy(decodedFrame->GetData(), mCurrentFrame->data[0], ySize);
            }

            // Copy V plane, but linewise
            for (int v = 0; v < mCurrentFrame->height; v += 2)
            {
                memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[2] + mCurrentFrame->linesize[2] * v, mCurrentFrame->width / 2);
                offset += mCurrentFrame->width / 2;
            }

            // Copy U plane, but linewise
            for (int u = 0; u < mCurrentFrame->height; u += 2)
            {
                memcpy(decodedFrame->GetData() + offset, mCurrentFrame->data[1] + mCurrentFrame->linesize[1] * u, mCurrentFrame->width / 2);
                offset += mCurrentFrame->width / 2;
            }
        }

mCurrentFrame is the normal AVFrame structure from FFmpeg.

I still think it might be a padding issue, but any help would be much appreciated!

0

There are 0 best solutions below