What Is Reprojection Error in Camera Calibration?

Reprojection error is the one number that shows whether a calibration reproduces the geometry it was fitted to. Depth, hand pose and object pose are all computed from pixel positions, so a camera model that misplaces known points by a large margin will misplace everything else too. A buyer who is shown calibration parameters without an error figure has no way to judge them.

Each episode carries a calibration/ group with intrinsics, extrinsics (SE3) and rms_reproj_px, solved before and after capture. The site states that an episode ships only if RMS reprojection stays under 0.28 px and the drift between the opening and closing solve is within tolerance; otherwise it goes to the reject log. Both solves ship with the episode, so you can re-run the reprojection check on your own copy. Per-episode values, the calibration target type and the number of views are not stated.

Calibration works backwards from known geometry. You show the camera a pattern whose point positions are known, detect those points in the image, and solve for the parameters that explain what was seen. Reprojection error closes the loop: take the solved parameters, project the known 3D points back onto the image plane, and measure how far each projected point lands from the point the detector actually found.

The unit is the pixel, because image points are pixel coordinates in OpenCV’s pinhole model. Zero would mean the model reproduces every detection exactly. OpenCV’s calibration tutorial puts it plainly: reprojection error gives a good estimation of how exact the found parameters are, and the closer it is to zero, the more accurate they are. A nonzero value is expected, since detected corners are measured rather than exact.

The tutorial’s method has three moves. Given the intrinsic, distortion, rotation and translation parameters, transform each object point to an image point with projectPoints. Then take the absolute norm between that projection and the result of the corner-finding algorithm. Finally, average over the calibration images: the tutorial takes the arithmetical mean of the per-image errors.

OpenCV reports the same idea in two summaries. calibrateCamera returns the overall RMS re-projection error and can output perViewErrors, the RMS error for each pattern view. An RMS figure weights large misses more heavily than a mean of norms, so the two are not the same number even on the same data. Compare like with like, particularly against a threshold written as RMS.

Calibration needs several views of the pattern. The tutorial asks for at least 10 test patterns, and its corner-refinement step (cornerSubPix) exists to tighten the detected points the error is measured against.

OpenCV gives a direction, not a limit: closer to zero is better. Where to draw the acceptance line is a decision for whoever ships or buys the data, and it should be written down together with how the figure is computed.

The site’s stated line is narrow and should be read exactly. An episode ships only if RMS reprojection stays under 0.28 px and the drift between the opening and closing solve is within tolerance. Do not read 0.28 px as a universal standard, and do not read it as a depth accuracy: the site states no accuracy beyond that gate.

A low figure is also not the whole story. OpenCV warns that a failed distortion estimate may look deceptively good near the image centre while working poorly elsewhere, so read the error alongside a check on the distortion result. See lens distortion for how the coefficients are estimated.

OpenCV’s stereoCalibrate finds the intrinsic parameters of each of the two cameras and the extrinsic parameters between them. Like calibrateCamera, it minimizes the total re-projection error for all points in all views from both cameras, and it returns the final value. One number therefore pools two lenses and their relative pose.

That makes it the first check on a stereo rig, and it has limits worth knowing. The pooled value does not say how the error splits between the left and right camera; perViewErrors gives a per-view breakdown. It is also not an epipolar error, a rectification error or a depth error. Those come downstream, after undistortion and rectification, and the site states none of them.

Depth from a stereo pair also depends on the baseline, the fixed distance between the two lenses. The site describes its head-mounted stereo pair as set at a human interpupillary baseline and does not state the length. See stereo baseline for how disparity becomes depth.

The site states that stereo depth is only as trustworthy as the calibration behind it, and that a worn rig flexes during a session. A single up-front calibration can silently decay, and you find out only when the geometry stops lining up.

A reprojection figure describes how well one solve fits its own views. It says nothing about change between two solves. That is why the site pairs the reprojection gate with a drift check, bracketing every episode with an opening and a closing solve and carrying the measured drift as calib_drift_mm in the episode metadata.

Firsthand runs fast, on-demand custom collections against a written spec, so the requirement belongs in the spec. State the threshold, whether it is RMS or a mean, whether it applies per camera, per solve or pooled, and which calibration fields must ship with every episode.

Because both solves ship, you can re-run the check on a pilot batch before committing to volume. Write any tighter threshold or any rectification check into the spec rather than assuming it, then request a custom dataset.

What is reprojection error in camera calibration?

It is the pixel distance between a detected calibration point and the position the calibrated camera model projects for the same 3D point. OpenCV says the closer it is to zero, the more accurate the found parameters are.

What is a good reprojection error?

OpenCV states no threshold, only that closer to zero is better. The acceptance line is a policy choice. The site’s stated gate is RMS reprojection under 0.28 px with drift within tolerance, and it is not extended beyond that.

Is reprojection error the same as stereo calibration accuracy?

No. In OpenCV’s stereoCalibrate it is the total re-projection error over both cameras, which makes it a first check on the rig. Epipolar, rectification and depth errors are separate downstream measurements.

Is a low reprojection error enough to trust depth data?

Not by itself. It describes how well one solve fits its own views and says nothing about drift between solves. The site pairs the reprojection gate with an opening-to-closing drift check and states no depth accuracy derived from it.

What reprojection error does Firsthand state?

An episode ships only if RMS reprojection stays under 0.28 px and the drift between the opening and closing solve is within tolerance. Per-episode values and the per-camera split are not stated.

Can I set a reprojection requirement in a custom collection?

Yes. Firsthand runs fast, on-demand custom collections against a written spec, so state the threshold, how it is computed and the fields that must ship, then request a custom dataset.