Camera Geometry: How can I determine the operating range of a stereo depth camera?

Question

I have been working on a project for the past few weeks to create a 3D point cloud with my custom stereo depth camera.

What I’m curious about is how to determine the operating range of a stereo depth camera. I want to make a stereo depth camera that operates in the range 0.5m~10m.

The equation I know is as follows:

Z = (Baseline * Focal length) / Disparity

• The focal length of my two camera: 935 pixels
• The field of view of my two camera: 108.4°(H) x 92.2°(V) x 120°(D)
• The resolution of my two camera: 2592 x 1944
• The baseline of my two cameras: ???
• The disparity of my two cameras: ???

How should I determine the baseline and disparity to make a stereo depth camera that operates in the range of 0.5m~10m?

To generate a depth map from images taken by two cameras, I’m using the python opencv stereoSGBM algorithm. Is the disparity in equation 1 determined by the numDisparity that goes into the parameters of the stereoSGBM function?

I’m asking this because I can’t find related materials ☹

Answer 1

Baseline: the physical distance between those "eyes". Grab a ruler and eyeball it or browse the specs of your stereo rig. For determining a good baseline given performance requirements, see below.

Disparity, unit "pixels", maybe a real number, is a per-pixel value that's coming out of block matching or other calculations. For points at infinity, it's supposed to be 0. It becomes large for near points.

For your estimation, you only need to know if your disparity is estimated in whole pixels or with any subpixel precision. Subpixel accuracy comes from block matching, where the block matching is an implicit lowpass (smoothing) that buys you the subpixel accuracy.

For StereoSGBM, you can assume 1/16 subpixel resolution (i.e. 4 fractional bits). If the algorithm gives you a disparity value of 16, that means 1.0 pixels of disparity. It's either mentioned somewhere in the docs or you'll have to discover this factor from example code or the source of the actual algorithm. For StereoSGBM, you have mindisp and numdisp, right? then maxdisp = mindisp + numdisp. Those are given in whole pixels.

When you plot the distance for every possible disparity value from 0 to whatever maximum (np.arange), you get the "steps" in your depth map. Then you can consider if that resolution is sufficient within the working range you consider. This will also tell you the (maximum) disparity for the near end of your working range.

If you start with a depth range and a minimum resolution, you can calculate backwards to arrive at a required baseline.

Also be sure you're passing the two pictures in the right order, or else you'll get "noise".

Always visualize the disparity map values. The map may contain negative values, which signify "can't find a block match".

baseline = 80e-3 # [m] 80 mm
f = 935 # [px]
disps = np.arange(0, 150, 1/16) # [px] evaluating disparity of 0 to 150 pixels
Z = baseline * f / disps # [m]
Z_step = np.diff(Z)
np.vstack([disps[:-1], Z[:-1], Z_step]).T

array([[   0.     ,        inf,       -inf],
       [   0.0625 , 1196.8    , -598.4    ], # 1/16 px, Z in meters
       [   0.125  ,  598.4    , -199.46667],
       ...
       [   7.4375 ,   10.05714,   -0.08381],
       [   7.5    ,    9.97333,   -0.08242],
       ...
       [ 149.5625 ,    0.50013,   -0.00021],
       [ 149.625  ,    0.49992,   -0.00021],
       ...

So you're getting an ~83.8 mm step near 10 meters distance, and a ~0.21mm step near 0.5m.

That is a factor of 400 in the Z resolution, for a factor of 20 in your Z range (10 to 0.5 meters).

Disparity around the near end of your range will get quite large, and insanely finely resolved. You should consider if you need that. The block matcher's work is measured in disparity, not distance. It will spend a lot of work testing for block matches closeby, and very little time testing far matches.

I was trying to come up with a closed form equation to calculate required baseline given some Z resolution at some Z distance but don't have the concentration right now. I'm sure you can get there, or with trying various values and checking what the forward calculation gives you.