Glossary
What Is Camera Calibration?
Last reviewed: September 2026
Short answer
Camera calibration is the process of solving a camera’s intrinsic parameters (lens and sensor properties like focal length and distortion) and, for a multi-camera rig, its extrinsic parameters (the 3D pose of each camera relative to the others). For a stereo pair, calibration also includes the baseline — the fixed distance between the two lenses — which is what lets disparity be converted into metric depth.
01
Why it matters
Uncalibrated or drifted camera parameters make depth, hand pose, and object pose unreliable, because triangulation and rectification depend on knowing exactly where each camera is and how its lens distorts the image. A rig that is calibrated once and assumed to stay fixed can silently drift over a capture session, corrupting geometry exactly when a policy most needs it to be accurate.
02
How Firsthand handles it
Firsthand brackets every episode with two calibration solves — one before capture and one after — for a head-mounted global-shutter camera plus two wrist cameras and a depth sensor. Intrinsics (fx, fy, cx, cy, and lens distortion coefficients) and extrinsics (a 4×4 SE(3) pose per camera, with the head-camera frame as identity) are solved each time, and an episode ships only if RMS reprojection stays under 0.28 px and the drift between the opening and closing solve is within tolerance. See camera calibration procedure for the full step-by-step.
03
What camera calibration solves
| Parameter | What it captures | How Firsthand handles it |
|---|---|---|
| Intrinsics | fx, fy, cx, cy, k1, k2, p1, p2, k3 per camera | Solved per episode, before and after capture |
| Extrinsics | 4×4 SE(3) pose per camera; head-camera frame is identity | Re-solved on the closing pass to measure drift |
| Stereo baseline | Fixed distance between the two lenses in a stereo pair | Measured every session and shipped with the episode |
| Depth alignment | Depth registered to the head-camera frame | Uses the same solved extrinsics as the video streams |
| Acceptance gate | Whether the calibration is trustworthy enough to ship | RMS reprojection under 0.28 px, drift within tolerance |
05
Frequently asked questions
- What is camera calibration?
- The process of solving a camera’s intrinsic parameters (lens and sensor properties) and, for a multi-camera rig, its extrinsic parameters (the 3D pose of each camera relative to the others), so pixels can be converted into accurate geometry rather than assumed.
- What is the difference between intrinsics and extrinsics?
- Intrinsics describe a single camera’s own lens and sensor properties — focal length, optical center, and distortion coefficients. Extrinsics describe where that camera sits in 3D space relative to the other cameras in the rig.
- What is a stereo baseline in camera calibration?
- The fixed distance between the two lenses in a stereo camera pair. It is one of the calibration values — alongside intrinsics and extrinsics — that lets disparity between the left and right images be converted into metric depth. See stereo baseline for the full explanation.
- How often does Firsthand calibrate its cameras?
- Every episode is bracketed by two calibration solves — one before capture and one after — rather than calibrating once and assuming the rig stays fixed. The difference between the two solves is the measured drift for that episode.
- What happens if calibration drifts during an episode?
- The closing solve measures the drift against the opening solve. An episode is only accepted if RMS reprojection stays under 0.28 px and the drift is within tolerance; otherwise it goes to the reject log. See camera calibration procedure for the full gating logic.
- How is depth affected by camera calibration?
- Depth is registered to the head-camera frame using the same solved extrinsics as the video streams, so accurate calibration is what keeps depth geometrically aligned with the RGB and pose data captured alongside it.
Related terms