What Is Lens Distortion in Camera Calibration?

Stereo depth, hand pose and object pose all assume rays map to pixels the way the camera model says. Lens distortion breaks that assumption, most strongly toward the frame edge. A buyer who cannot tell whether distortion was modelled, and by which coefficients, cannot tell whether depth and pose labels can be trusted.

The site states per-camera intrinsics (fx, fy, cx, cy, k1, k2, p1, p2, k3) in each episode’s calibration/ group, solved before and after capture, with rms_reproj_px under 0.28 gating shipment. Coefficient values, lens model and field of view are not stated.

The ideal pinhole camera model projects every straight line in a scene as a straight line in the image. OpenCV, the reference implementation this page follows, notes that real lenses usually have some distortion, mostly radial distortion and slight tangential distortion, so its calibration model extends the pinhole model with extra terms.

The effect is geometric, not a matter of sharpness or colour: a point lands at a different pixel than the pinhole model predicts. OpenCV states that distortion coefficients do not depend on the scene viewed, which is why they are estimated once per camera and treated as intrinsic parameters.

Radial distortion makes straight lines look curved and grows with distance from the image centre. OpenCV names two common forms: barrel distortion, where the radial scaling factor decreases away from the centre, and pincushion distortion, where it increases.

Tangential distortion occurs because the lens is not aligned perfectly parallel to the imaging plane, so some areas of the image look nearer than expected. OpenCV calls it slight on real lenses. Both sit in the same model, so one calibration solves for them together.

OpenCV’s standard distortion vector is (k1, k2, p1, p2, k3). The k terms describe radial distortion and p1 and p2 describe tangential distortion. Extended models add k4 to k6, thin-prism terms s1 to s4 and tilt terms, and a four-element vector means k3 is zero.

They are estimated during intrinsic calibration. OpenCV’s calibrateCamera takes known pattern points and their detected image points from several views and returns the camera matrix and distortion coefficients; its tutorial asks for at least 10 pattern images. Quality is judged by re-projection error, where closer to zero means more accurate parameters, and OpenCV says a non-monotonic radial result should be treated as a calibration failure.

Unlike fx, fy, cx and cy, the coefficients stay the same when image resolution changes.

Rectification warps both images so matching points share a row, and it assumes each image follows the pinhole model. OpenCV’s tutorial is direct: for stereo applications, these distortions need to be corrected first. Left in, the bending displaces points in a way the rectification transform does not model, and the disparity found along a row is wrong.

OpenCV’s stereoRectify takes each camera’s distortion coefficients as input, and initUndistortRectifyMap computes one map that undistorts and rectifies together. The order is calibrate, undistort, rectify, match, then convert disparity to depth. OpenCV adds that undistorting with alpha 0 can remove pixels at the image corners, while alpha 1 keeps all pixels with some black borders.

Hand pose, object pose and depth are computed from pixel positions. The site’s camera calibration entry states that uncalibrated or drifted parameters make them unreliable, because triangulation and rectification depend on knowing how each lens distorts the image. Radial distortion is largest toward the frame edge, so geometry measured there is affected most.

The site also states that each camera on the rig, head and wrist, gets its own independent intrinsic solve, so distortion is modelled per camera rather than assumed. For a buyer, the checks are whether the delivery carries those per-camera coefficients and whether your own pipeline can reproduce the undistortion.

Each episode carries a calibration/ group with intrinsics, extrinsics (SE3) and rms_reproj_px. Intrinsics are listed as fx, fy, cx, cy, k1, k2, p1, p2, k3 per camera. The rig is a head-mounted global-shutter camera plus two wrist cameras and a depth sensor. Calibration is solved before and after capture, and an episode ships only if RMS reprojection stays under 0.28 px and open-to-close drift is within tolerance.

Raw frames ship with calibration so you can re-rectify, and rectified pairs plus disparity can be added to the delivery. The stereo baseline is described only as human interpupillary. The site does not state the lens model, field of view, distortion model name, coefficient values or residual distortion. To require any of them, write it into the spec for a custom or on-demand collection.

What is lens distortion?

It is how a real lens bends straight lines away from the ideal pinhole model, so points land at different pixels than predicted. It is mostly radial, with slight tangential distortion.

What is the difference between radial and tangential distortion?

Radial distortion curves straight lines and grows with distance from the image centre. Tangential distortion comes from the lens not sitting perfectly parallel to the imaging plane.

What are distortion coefficients?

They are the numbers that model the bending. OpenCV’s standard set is k1, k2, p1, p2 and k3, estimated from several views of a calibration pattern during intrinsic calibration.

Why undistort before stereo rectification and depth?

OpenCV’s tutorial says stereo applications need distortion corrected first. Rectification and disparity assume pinhole geometry, so uncorrected bending carries into depth.

Do distortion coefficients change with image resolution?

No, per OpenCV they stay the same regardless of resolution. The focal lengths and principal point, fx, fy, cx and cy, must be scaled when the image is scaled.

Does Firsthand state its distortion values or lens model?

Not stated. The site lists the intrinsic fields (k1, k2, p1, p2, k3) and the 0.28 px reprojection gate, but not coefficient values, lens model, field of view or residual distortion.

Can I request distortion requirements in a custom collection?

Yes. Firsthand runs custom and on-demand collections against a written spec, so state the calibration fields, delivery form and checks you need, then request a custom dataset.