Main Street Autonomy

Common camera problems

The following artifacts arise when cameras operate on moving platforms and in uncontrolled lighting.

Rolling-shutter skew and wobble

A rolling shutter exposes the sensor row by row, so a fast pan causes vertical edges to lean (skew) and vibration shears the frame (wobble, or jello). It also corrupts geometry: feature positions are sampled at different times down the frame, which biases visual odometry and stereo.

Rolling-shutter skew on a moving vehicle
Figure 15 — Rolling-shutter distortion: a vehicle photographed from a moving car is sheared because each row is exposed at a slightly different time. Source: Axel1963, CC BY-SA 3.0, via Wikimedia Commons.

Lens flare and veiling glare

Bright sources such as the sun or headlights scatter inside the lens, producing flare artifacts and a low-contrast haze (veiling glare) across the frame. Contrast falls when the camera points toward a low sun. Flare is fundamentally an optical phenomenon rather than a property of the sensor.

Lens flare from the sun behind a monument
Figure 16 — Lens flare and veiling glare from the sun within the frame; scattered light adds streaks and a low-contrast haze. Source: W.carter, CC BY-SA 4.0, via Wikimedia Commons.

Chromatic aberration

The lens focuses different wavelengths at slightly different points, leaving color fringes on high-contrast edges. The effect is worst at the frame periphery and on wide lenses.

Purple fringing from chromatic aberration
Figure 17 — Chromatic aberration: purple and magenta fringes around bright highlights, where the lens focuses different wavelengths at slightly different points. Source: Jarmniku, CC BY-SA 4.0, via Wikimedia Commons.

Vignetting

The frame is darker toward the corners, caused by optical falloff and the lens aperture. It biases photometric methods and any algorithm that assumes uniform brightness across the frame.

Bloom and saturation

A strong return or a direct light source saturates pixels and bleeds charge into neighboring pixels (bloom), smearing bright regions. Retroreflectors and the sun are common causes. Beyond saturation a pixel carries no further information.

Auto-exposure hunting

Auto-exposure reacts over several frames, so rapidly changing light, such as dappled shade or the mouth of a tunnel, causes brightness to oscillate and leaves a sequence of over- or under-exposed frames at each transition.

Thermal focus drift

Temperature changes alter lens and mount dimensions, shifting focus and slightly changing the intrinsics. Outdoor and engine-bay cameras drift between a cold start and steady-state operation. A calibration captured at one temperature does not hold across a wide temperature range.

Depth-specific failures

  • Textureless stereo dropouts: passive stereo returns gaps on blank walls, sky, and smooth floors, where no disparity can be computed.
  • Structured-light sunlight washout: the projected pattern is overwhelmed by sunlight, so structured light degrades outdoors.
  • iToF multipath and flying pixels: indirect time-of-flight biases range in concave geometry (multipath) and produces points that float between foreground and background at depth discontinuities (flying pixels). See Depth.