When decoding a jpeg how would I reverse chrominance downsampling in matlab?

Question

Hi trying to make a simple jpeg compressor which also decompresses the image. I use the following code to downsample the chrominance of an image for the first step of jpeg compression.

resampler = vision.ChromaResampler;
[Cb, Cr] = resampler(Cb_channel, Cr_channel);

the function is part of the computer vision toolbox for matlab.

for example before downsampling:

Y dimensions = 3024 by 4032; Cb and Cr dimensions = 3024 by 4032

after downsampling:

Y dimensions = 3024 by 4032; Cb and Cr dimensions = 3024 by 2016

to display the original RGB image after decompression, the dimensions of all 3 Y, Cb and Cr components need to be the same so that I can merge the channels and convert the image back to RGB. I'm using the following code to achieve this:

Cb_resized = imresize(Cb, [(size(Cb, 1)) (2*size(Cb, 2))]);
Cr_resized = imresize(Cr, [(size(Cr, 1)) (2*size(Cr, 2))]);

When I then merge the 3 channels and use imshow() to see the image it looks fine. So is the above method the correct way of reversing the downsampled chrominance when decoding a jpeg?

Answer 1

Using imresize for up-sampling is "almost correct".

Instead of using imresize, you better use vision.ChromaResampler for up-sampling:

up_resampler = vision.ChromaResampler();
up_resampler.Resampling = '4:2:2 to 4:4:4';

[Cb_resized, Cr_resized] = up_resampler(Cb, Cr);

The module is design such that Resampling = '4:2:2 to 4:4:4' "reverses" the result of Resampling = '4:4:4 to 4:2:2'.

---

The '4:4:4 to 4:2:2' ChromaResampler uses a convention that displaces the result 0.5 pixels to the right.
(I think shifting by 0.5 pixels supposes to match MPEG-1 codec standard).

The 0.5 displacement is not well documented - I had to build a short test for figuring it out.

As far as I remember, the convention of moving 0.5 a pixel is used by MPEG-1 codec, but not used by MPEG-2 and newer codecs.
I don't think it is used by JPEG, but I am not sure...

Note:
Since the human visual system is not very sensitive to the Chroma resolution, you are probably not going to see the differences if 0.5 displacement is used or not.

---

For getting the same result as ChromaResampler, you may use imwarp, with displacement of 1 pixel in the horizontal axis.

Understanding imwarp is a little complicated.
I am going to use imwarp for demonstrating that 1 pixel displacement, gives the same result as ChromaResampler:

The following code sample shows the equivalence:

close all

I = imread('peppers.png'); % Read sample image
YUV = rgb2ycbcr(I); % Convert RGB to Y:Cb:Cr
U = YUV(:, :, 2); % Get U color channel
V = YUV(:, :, 3); % Get V color channel

down_resampler = vision.ChromaResampler(); % 4:4:4 to 4:2:2
down_resampler.Resampling = '4:4:4 to 4:2:2';

up_resampler = vision.ChromaResampler(); % 4:2:2 to 4:4:4
up_resampler.Resampling = '4:2:2 to 4:4:4';

% Down-sample U and V using ChromaResampler
[downU, downV] = down_resampler(U, V);

%downU2 = imresize(U, [size(U, 1), size(U, 2)/2]); % Not the same as using imresize
%figure;imshow(downU);figure;imshow(downU2);

% Up-sample downU and downV using ChromaResampler
[upU, upV] = up_resampler(downU, downV);

% Result is not the same as using imresize
%resizedU = imresize(downU, [size(downU, 1), size(downU, 2)*2], 'bilinear');
%resizedV = imresize(downV, [size(downV, 1), size(downV, 2)*2], 'bilinear');

% Use transformation matrix that resize horizontally by x2 and include single pixel horizontal displacement.
tform = affine2d([ 2   0   0
                   0   1   0
                  -1   0   1]);

% Use imwarp instead of imresize (the warp includes horizontal displacement of 1 pixel)
warpU = imwarp(downU, tform, 'bilinear', 'OutputView', imref2d([size(downU, 1), size(downU, 2)*2]));
warpU(:, end) = warpU(:, end-1); % Fill the last column by duplication

%figure;imagesc(double(upU) - double(resizedU));impixelinfo
%figure;imshow(upU);figure;imshow(resizedU);
%figure;imshow(upU);figure;imshow(warpU);

% Show the differences:
figure;imagesc(double(upU) - double(warpU));title('Diff');impixelinfo
max_abs_diff = max(imabsdiff(warpU(:), upU(:)));
disp(['max_abs_diff = ', num2str(max_abs_diff)]); % Maximum absolute differenced is 1 (due to rounding).

Note: The imresize usage is kept in comments.

---

Note:
The default interpolation method of imresize is cubic interpolation, and the default interpolation method of ChromaResampler is linear interpolation.
Cubic interpolation is considered superior, but linear interpolation is commonly used (the visible difference is negligible).