How does resizing an image affect the camera calibration projection matrix?

Question

I am presently working on Camera calibration methodology. I am very new to this field. Given a set of 2D-3D correspondence, the least-square fitting approach is used to construct the 3*4 projection matrix.

Case - 1: I have a taken an image $I_1$ with size $H$ x $W$ and computed the $3*4$ projection matrix $M_1$. Case - 2: I have resized the image ($I_1$) with 2x interpolation and got the new Image $I_2$ with size $2H$ x $2W$ and then computed the projection matrix $M_2$.

What is the relation between $M_1$ and $M_2$. Are they same or different ? If same, why? If different, what is the mathematical relationship between $M_1$ and $M_2$ ?

I have experimentally seen that both the outputs $M_1$ and $M_2$ are approximately same (not exact, some deviation after second decimal). I am not sure, whether, I am approaching right or not.

Answer 1

Your question lack some code for a precise answer. It depends on how the matching is done.

For example, it is quite usual to consider matrix that are "dimension-less", that is considering that image coordinates goes from 0 to 1 (or -1 to 1, with center at center of image). In which case, image resolution doesn't matter.

If you consider the case where image coordinates are the traditional index in H×W array (so starting with y=0, x=0 at top left corner, and ending with y=H, x=W at bottom right corner), then, resizing the image is the same as multiplying coordinates of points in this image by matrix

Z=[[ρ,  0,  0],
   [0,  ρ,  0],
   [0,  0,  1]]

So, a point that appears at coordinates (x,y,1) in the initial image, now appears at coordinates (ρx, ρy, 1) in the new one (with, in your example ρ=2)

Or, since in those homogeneous coordinates, (ρx,ρy,1) is the same as (x,y,1/ρ), it could as well be matrix

Z=[[1,  0,  0],
   [0,  1,  0],
   [0,  0,  1/ρ]]

(Which makes sense: anything, matrix or coordinates, are valid to a proportional factor)

So, if your 3×4 projection matrix is M, then a point in real life (xᵣ,yᵣ,zᵣ,1) is projected in the original image as (xᵢ,yᵢ,1) = M.(xᵣ,yᵣ,zᵣ,1), and once resized as (ρxᵢ,ρyᵢ,1) = Z(xᵢ,yᵢ,1) = ZM(xᵣ,yᵣ,zᵣ,1).

In other words, just multiply the two first lines of the matrix M by ρ. Or divide the last one by ρ, and you get the new projection matrix.

Again, if by "projection matrix" you mean the one that transforms coordinates of the real world into index in the 2D-array representing the image. If those coordinates are not index, but proportion ratio expressed in image size (0=center, 1=right, -1=left, or something like that), or real units, like centimeters, W and H being just the resolution, not the size of the image, then, projection matrix is not impacted in anyway by resolution of the image (its estimation can be, obviously: the better the resolution is, the better the estimation of the projection matrix)

Answer 2

The most common definition of 'projection matrix' in the context of camera calibration is:

M = K * [R|t], where

K = [[fx, 0, cx], [0, fy, cy], [0, 0, 1]].

Note: some literature mentions a pixel skew parameter, but in digital sensors, this can usually be neglected.

This matrix projects 3D points (homogenous world coordinates) into 2D pixel coordinates in the image (again as a homogenous vector). This convention is adopted by OpenCV, libCalib, Matlab, Halcon, and in the scientific literature on camera calibration and machine/computer vision. Sometimes, "normalized camera coordinates" are employed where points are simply projected onto the plane at Z=1, and the K matrix can be left out in the formula above. When rescaling an image by a factor s, a new, corresponding K matrix can be derived. Its form is dependent on the coordinate convention employed:

For OpenCV, Calib.io Calibrator, Halcon, the pixel origin is at the center of the top-left pixel. The new K matrix becomes:
Ks = [[s*fx, 0, s*(cx+0.5)-0.5], [0, s*fy, s*(cy+0.5)-0.5], [0, 0, 1]]

If the pixel origin is at the top left corner of the top left pixel:

Ks = [[s*fx, 0, s*cx], [0, s*fy, s*cy], [0, 0, 1]]

Mor information can be found in this article which I have authored: https://calib.io/blogs/knowledge-base/camera-models