I am looking to convert CVPixelBuffers from one of the Biplanar YUV color spaces (YUV420, YUV422, YUV4444) to another using Metal compute or fragment shaders. Here is what I have understood so far:
To convert from YUV422 to 420, I can downsample by either skipping alternate UV values vertically, or I can take an average of two vertical UV values. Some softwares such as FFMpeg are also known to do some kind of antialiasing.
To convert from YUV420/422 to YUV4444, one needs to perform upscaling using linear or cubic interpolation.
I don't know the best approach for doing these conversions on live video frames using Metal. Core Image already seems to implement these color space conversions, but being closed source it is not clear how it performs conversion and also CPU utilization takes a big hit when using Core Image instead of Metal.