What Is SLAM for Egocentric Video?

In egocentric video the camera moves with the wearer’s head, so hand positions are measured relative to a moving frame. EgoScale’s paper gets the world-frame wrist pose by composing the estimated camera pose with the hand pose in the camera frame, so a camera-pose estimate sits inside the action signal. The tradeoff is accuracy: the paper states the estimates are "noisy due to unconstrained data collection," and relies on scale and diversity to make them useful supervision.

Firsthand does not run SLAM and does not state that it delivers a head or camera pose trajectory. Its rig records calibrated head and wrist cameras, active-stereo depth, 200 Hz 6-DoF IMUs and 3D hand pose on one PTP clock. Collection is fast, on demand, to a written spec.

In EgoScale’s paper, SLAM is one of two off-the-shelf perception pipelines run on head-mounted egocentric RGB video: "We apply off-the-shelf SLAM and hand-pose estimation pipelines to recover camera motion and human hand trajectories."

The estimated camera pose is represented as an SE(3) transform between the camera and world frames. The recordings are RGB at 30 FPS. Which SLAM pipeline was used is not stated.

The paper models hand pose as 21 keypoints in the camera frame. To place the wrist in the world frame it composes that with the estimated camera pose.

To get motion commands "invariant to global camera movement," it then uses relative wrist motion between consecutive timesteps, which it says "removes dependence on absolute camera pose."

EgoScale uses them to label its large egocentric corpus. This site’s EgoScale pages report 20,854 hours of action-labeled egocentric human video, with labels from off-the-shelf SLAM and hand-pose estimation.

The paper concedes the estimates are noisy, yet states "the scale and diversity of the data provide effective supervision for learning transferable action representations."

They are separate estimates that combine. SLAM supplies camera motion, hand-pose estimation supplies the hand keypoints, and together they yield a wrist-level action representation shared across human demonstrations and robot executions.

EgoScale also uses a smaller, embodiment-aligned human dataset where Vive trackers give wrist pose and Manus gloves record 25 joint transforms, all synchronized with the video stream.

Firsthand states a head-mounted global-shutter 4K/60 camera, two wrist 1080p/60 cameras, an active-stereo depth sensor at 848×480/30 and 200 Hz 6-DoF IMUs, hardware-triggered onto one PTP clock. Intrinsics and extrinsics are re-solved with a checkerboard at the start and end of every session.

3D hand pose is delivered as handJoints[2][21][3] in metric units in the head-camera frame, with a visibility flag per joint. A per-frame world-frame camera trajectory is not stated on this site.

When coverage of a specific scene, task or object set is thin, or a program needs commercial rights, labels recovered from unconstrained public video do not fill the gap. EgoScale shows what off-the-shelf pipelines can do at scale, not what any one program lacks.

Firsthand’s custom collection targets that gap: fast, on-demand capture to a written spec, delivered with a buyer-owned commercial license.

What does SLAM do for egocentric video?

It recovers camera motion from the footage. EgoScale (arXiv 2602.16710) applies off-the-shelf SLAM and hand-pose estimation to recover camera motion and human hand trajectories.

Why does egocentric robot pretraining need camera pose?

Hands are measured in the camera frame and the camera moves with the head. EgoScale composes the estimated camera pose with hand pose for a world-frame wrist pose.

Are SLAM-derived action labels accurate?

The paper calls the estimates "noisy due to unconstrained data collection" and relies on scale and diversity for effective supervision. It does not state SLAM accuracy figures.

How does SLAM relate to hand-pose estimation?

They are separate pipelines. SLAM gives camera motion and hand-pose estimation gives hand keypoints. EgoScale combines both into a wrist-level action representation.

Does Firsthand run SLAM or deliver camera pose?

No. Firsthand does not run SLAM and does not state a head or camera pose trajectory. It delivers calibrated streams, depth, 200 Hz IMUs and 3D hand pose on one PTP clock.

What does Firsthand record instead of SLAM output?

Head and wrist video, active-stereo depth, 6-DoF IMUs and handJoints[2][21][3] with visibility flags, plus intrinsics and extrinsics, captured fast and on demand to a written spec.