FAQ

Should I use a color or monochrome camera?

Use monochrome for geometry: stereo, visual odometry, and feature tracking. A mono sensor collects two to three times more light than the same sensor with a Bayer filter, has no demosaicing artifacts, and resolves finer detail. Use color images for perception: object detection and tracking, classification, scene segmentation, etc.

Global or rolling shutter?

MSA recommends rolling shutter imagers: they are cheaper, higher resolution, and have better low-light performance.

Does a depth camera replace lidar?

No. Stereo error grows with the square of range, structured light and ToF are short-range and sensitive to sunlight, and all depend on the IR source or scene texture. Lidar measures range directly to long range in sunlight. Depth cameras are dense, cheap, and detailed up close; lidar is sparse, long-range, and robust. Many autonomy stacks use both.

Why does my stereo depth get worse with distance?

Because depth is inversely proportional to disparity, so a fixed one-pixel matching error maps to a depth error that grows with the square of range. Widen the baseline, increase focal length or resolution, or use subpixel disparity to push the usable range out. See Depth.

Why are my two cameras giving wrong depth even though each looks fine?

Almost always due to synchronization or calibration problems. Unsynchronized frames put the two views at different instants, which is a large disparity error on a moving robot; use hardware sync. If sync is good, recheck the intrinsics, extrinsics, and baseline.