Image sensors and shutter

A camera’s imaging sensor converts photons to a digital value per pixel. The underlying pixel technology and how the pixels are exposed and read out (the shutter) are two important factors.

CMOS vs CCD

Almost every robotics camera uses a CMOS active-pixel sensor. CMOS puts an amplifier and often an ADC on the sensor die, which gives high frame rates, region-of-interest readout, low power, and low cost. Older CCD sensors shift charge across the chip to a single readout node, which gives clean images but is slow, power-hungry, and hard to integrate. CCD is effectively obsolete for robotics.

What determines image quality

• Pixel size: a larger pixel collects more photons, so it has a better signal-to-noise ratio and dynamic range. Small pixels improve resolution at the cost of low-light performance.
• Quantum efficiency (QE): the fraction of incident photons that produce signal. Higher QE means more usable light, which matters in the dark and at short exposures.
• Read noise and full-well: read noise sets the noise floor; full-well capacity sets the ceiling. Their ratio bounds dynamic range.
• Back-side illumination (BSI): moves the wiring behind the photodiode so more light reaches it. Standard on small-pixel sensors.
• Stacked sensors: bond the pixel array to a separate logic wafer (Sony calls this Exmor RS), which frees area for faster readout and on-chip processing.
• Binning: combine neighboring pixels to trade resolution for sensitivity and speed.

Rolling vs global shutter

A rolling shutter exposes and reads the sensor row by row. Each row starts its exposure slightly after the row above, so the top and bottom of a frame are captured at different times. On a moving robot this produces:

• Skew: straight vertical edges lean when the camera pans.
• Wobble (jello): the image shears under vibration.
• Partial exposure under pulsed light: LEDs and IR projectors that strobe can band the image, since different rows see different parts of the pulse.

A global shutter exposes every pixel at once, then reads out the data. No skew, no jello, and clean behavior with pulsed illuminators and multi-camera sync. The downside is a more complex sensor that requires per-pixel storage, which can mean larger pixels, lower resolution, and higher price.

Teledyne has a good technical explanation of rolling vs global shutter.

Common robotics sensors

Note the pattern: high-resolution color modules tend to be small-pixel rolling shutter, while the sensors used for stereo and visual odometry are lower-resolution global shutter with larger pixels. That is the resolution-versus-motion tradeoff made concrete.