Depth

Depth cameras measure range by one of three methods:

• Triangulation between two views (stereo)
• Projection of a known pattern followed by triangulation against it (structured light)
• Measurement of the light’s round-trip time (time-of-flight).

Each method has a distinct operating regime and characteristic failure modes.

Passive stereo

Two cameras separated by a known distance observe the scene from slightly different viewpoints. A surface point appears at different image columns in the two views, and that shift is the disparity d. With focal length f (pixels) and baseline B (the separation between the cameras), depth is:

Depth is inversely proportional to disparity, which determines the principal characteristics of stereo:

• Range and error. A fixed disparity error of one pixel corresponds to a depth error that scales with Z². Stereo is precise at short range and degrades quadratically with distance.
• Baseline. A wider baseline produces larger disparity at a given depth, improving far-range precision, and raises the minimum range below which the two views no longer overlap.
• Texture. Matching requires distinguishable detail. Uniform surfaces such as blank walls, sky, and smooth floors produce no disparity, leaving gaps in the depth map.

Computing depth requires calibration and rectification, which aligns the two images row-for-row so that a point in the left image lies on the same row in the right (the epipolar constraint), followed by a search along each row for the matching column.

Block matching is fast but noisy; semi-global matching (SGM/SGBM) adds smoothness penalties across multiple directions to produce a denser, cleaner disparity map at higher computational cost. OpenCV provides StereoBM and StereoSGBM.

Passive stereo cameras include the Stereolabs ZED 2/2i (120 mm baseline, approximately 0.2 to 20 m, with depth computed by a neural network in the SDK), the Intel RealSense D455 (95 mm baseline), and the Luxonis OAK-D (75 mm baseline, with on-device inference). Monochrome global-shutter pairs are typical, since matching benefits from sensitivity and edge sharpness rather than color.

Structured light

Structured light removes the texture requirement by projecting a known infrared pattern (dots or stripes) and triangulating against it, so depth no longer depends on scene texture. Two variants exist:

• Single-shot speckle: a fixed pseudo-random dot pattern is projected and matched against a stored reference. The original Microsoft Kinect (PrimeSense) used this method.
• Active stereo: a stereo pair is combined with an infrared projector that supplies texture; depth is still computed by stereo matching, with the projector filling in otherwise uniform surfaces. The Intel RealSense D415 and D435 use this approach.

The projected pattern competes with ambient light, so structured light performs well indoors and degrades in direct sunlight. The infrared pattern is invisible to color cameras but visible to other IR-sensitive cameras.

Time-of-flight

Time-of-flight (ToF) measures range directly from the travel time of light, per pixel, without a baseline or triangulation.

• Indirect ToF (iToF) modulates a continuous infrared source and measures the phase shift of the returned light to obtain range. It is dense and inexpensive, with three characteristic artifacts: phase wrapping, in which range becomes ambiguous beyond one modulation wavelength and requires unwrapping; multipath interference, in which light reaching a pixel by more than one path biases the range, particularly in concave geometry; and flying pixels at depth discontinuities, where a pixel mixes foreground and background returns into a point that lies between them. The Microsoft Kinect v2 (512×424 iToF) is the canonical example, and Sony’s DepthSense (IMX556) is a common iToF sensor.
• Direct ToF (dToF) times individual laser pulses per pixel using SPAD detectors. This is conceptually a flash lidar rather than a triangulation method; the Intel RealSense L515 packages a MEMS-scanned solid-state lidar in a camera form factor. For the underlying physics, see lidar, measuring distance.

Time-of-flight provides dense depth on textureless surfaces, but the infrared source limits range and largely restricts operation to indoor environments, and the artifacts above appear in the output.

Synchronization

Stereo and multi-camera depth algorithms require simultaneous frame capture. Two cameras nominally running at 30 fps can be offset by up to half a frame period, approximately 16 ms; on a moving platform this corresponds to a large disparity error and incorrect depth.

Hardware synchronization drives every sensor from a common trigger (FSYNC) so that exposures begin on the same edge, and a global shutter sensor shares a single exposure instant across the frame.

For multi-camera rigs with long cabling, GMSL/FPD-Link carries video and the synchronization signal together, and PTP (IEEE 1588) provides system-wide frame timestamps. The Stereolabs ZED X (GMSL2) and the Luxonis OAK-D provide hardware synchronization.

Calibration dependency

Every method above depends on known camera intrinsics and, for stereo, known extrinsics and baseline. Intrinsics are obtained from intrinsic calibration; the pose between cameras, and between a camera and a lidar or IMU, is obtained from extrinsic calibration.

Depth camera comparison

The three methods occupy different regimes: passive stereo operates outdoors and at longer range, structured light operates at close range on textureless indoor surfaces, and time-of-flight produces dense depth indoors. None matches lidar for range or for robustness in direct sunlight; see the FAQ.