What Is Epipolar Geometry in Stereo Video?

Stereo depth depends on finding the same point in both images. Epipolar geometry is what limits that search to a line, and it is only as reliable as the calibration behind it.

The site states a calibrated stereo rig with calibration/ intrinsics, extrinsics (SE3) and rms_reproj_px. It states no matrices, epipolar lines or epipolar error.

A single camera loses depth: every point along the ray through the camera centre lands on the same image point. A second camera sees those points at different positions, so the two views can be triangulated. Epipolar geometry is the set of rules that ties the two views together.

OpenCV’s epipolar geometry tutorial describes it with the camera centres, the epipolar plane and the epipoles.

The projection of the points along one camera’s ray forms a line in the other image. OpenCV calls it the epiline; searching along it instead of the whole image is the epipolar constraint.

The epipole is where the line through the two camera centres meets an image plane. All epipolar lines in an image pass through it, and it can fall outside the image when one camera does not see the other.

The essential matrix carries the rotation and translation of the second camera relative to the first. The fundamental matrix carries the same information plus both cameras’s intrinsics, so it works in pixel coordinates and maps a point in one image to an epipolar line in the other.

OpenCV estimates the fundamental matrix from matched points, with a minimum of 8 for the 8-point algorithm, and notes the result is sensitive to match quality and outliers. Known intrinsics and extrinsics determine both matrices directly, without estimating from matches.

OpenCV’s documentation states that stereoRectify computes rotation matrices that make both image planes the same plane, which makes all the epipolar lines parallel and simplifies dense stereo correspondence. For the horizontal layout, the lines become horizontal with the same y-coordinate.

Matching is then a one-dimensional row search, and the horizontal shift of the match is the disparity. The OpenCV stereo depth tutorial writes the depth relation as disparity = B·f / Z, with B the camera distance and f the focal length.

The site states a head-mounted, calibrated stereo rig at a human interpupillary baseline, described only qualitatively. Each episode carries a calibration/ group with intrinsics (fx, fy, cx, cy, distortion coefficients) per camera, extrinsics as a 4×4 SE(3) per camera with the head-camera frame as the identity, and rms_reproj_px. Calibration is solved at the opening and closing of each episode, and an episode ships only if rms_reproj_px stays under 0.28 and drift is within tolerance.

Raw frames ship with calibration, and rectified pairs plus disparity can be added on request. Depth is derived from the stereo rig as streams/depth_mm in metric millimetres, 480 × 848 at 30 Hz. The site states no fundamental or essential matrix, epipolar lines or epipoles as deliverables, so buyers derive them from the delivered calibration. The 0.28 figure is a reprojection gate, not an epipolar error.

Confirm every episode ships intrinsics, extrinsics and rms_reproj_px, with the reference frame documented. Both solves ship, so you can re-run the check.

Decide whether you will derive the geometry from raw frames or want rectified pairs added. The site states no epipolar or rectification figure, so write your own threshold into the spec and test it on a pilot batch.

What is epipolar geometry in simple terms?

It is the geometry between two cameras viewing one scene. A point in one image must lie on a matching line in the other, so finding a match means searching a line.

What is an epipolar line?

It is the line in one image on which the match for a point from the other image must lie. OpenCV calls it the epiline, and searching along it is the epipolar constraint.

What is the difference between the essential and fundamental matrix?

The essential matrix holds the relative rotation and translation of the cameras. The fundamental matrix adds both cameras’s intrinsics, so it maps a pixel in one image to an epipolar line in the other.

Why does rectification simplify stereo matching?

OpenCV states that it makes the epipolar lines parallel, which simplifies dense stereo correspondence. For the horizontal layout the lines become horizontal, so matching is a search along one row.

Does Firsthand deliver epipolar lines, matrices or rectified frames?

Not stated for lines or matrices. The site states intrinsics, extrinsics (SE3) and rms_reproj_px, raw frames with calibration, and rectified pairs plus disparity on request.

How accurate is the epipolar alignment?

Not stated. The stated check is rms_reproj_px under 0.28, which is a reprojection gate, not an epipolar or rectification error. Define your own check in the spec.

Can I request a custom collection with specific stereo deliverables?

Yes. State whether you want raw frames plus calibration or rectified pairs plus disparity, and the fields you expect. See calibrated egocentric video data, then request a custom dataset.