Humanoid Robot Data Collection Methods Compared

Humanoid programs draw training data from four sources: egocentric human video recorded with a head-worn camera rig, teleoperation where a human operator drives the physical robot, simulation that renders synthetic scenes and interactions, and licensed internet video. Each yields a different observation space, a different action representation, and a different fit for a given stage of training.

None of the four is a universal answer. A model that already has a broad visual and action prior from pretraining needs different data than one that is failing a specific task on its own hardware. Most humanoid data programs combine sources rather than picking one and stopping there.

Egocentric human video is recorded from a head-worn camera rig on a person performing a task, so the viewpoint matches what a humanoid perceives from its own head-mounted camera: hands entering frame from below, occlusion during grasp, motion during locomotion. A humanoid rig captures this alongside depth, an IMU, and 3D hand pose for both hands, so the two-arm action structure a humanoid needs is present directly in the recording rather than reconstructed afterward.

This method scales coverage across tasks, sites, and contributors faster than a robot-and-operator fleet can reach, because it does not need the physical robot to be present or available. Its limit is the embodiment gap: a human hand is not a robot end-effector, so a retargeting or alignment step is typically needed before the action content transfers to a robot policy.

Teleoperation records a human operator driving the robot directly, through direct teleop, VR or motion-capture teleop, or a leader-follower rig. It captures robot joint states and commands in the exact embodiment the policy will run on, with zero retargeting gap, because the "hand" doing the work is already the robot end-effector.

That embodiment match comes at a cost: teleoperation is bound by robot and operator availability, typically collects far slower per hour than human capture, and usually runs on a single robot platform at a time. It fits best when a small, tightly scoped skill set matters more than breadth of scenes or task variation, or when the policy must control a robot with zero tolerance for a retargeting gap.

Simulation renders synthetic scenes and interactions in a physics engine. It is cheap to scale and easy to vary in scene, object, and lighting count, but it carries a sim-to-real gap: a simulator’s approximation of physics, lighting, and materials does not match real-world contact dynamics exactly, so policies trained purely in simulation typically need real data to close that gap before deployment.

Licensed internet video offers broad visual variety at low marginal cost, but it rarely carries the structured labels a robot policy needs — 3D hand pose, depth, or action segments — and its license terms vary by source, so commercial use often requires separate clearance.

The embodiment gap is the mismatch between a human body and a robot’s hardware — different degrees of freedom, grasp geometry, and camera viewpoint — that keeps human demonstration data, including egocentric video, from transferring to a robot policy directly. Data captured from a human body does not map one-to-one onto robot joint commands without a retargeting or alignment step.

EgoScale’s published result is public evidence that this gap is solvable at scale: a small, aligned mid-training and post-training stage on real robot data, layered on top of large-scale egocentric pretraining, is what closes the gap and lifts real-robot task success, rather than any single data source doing the job alone.

Pretraining builds a broad visual and action prior and benefits most from egocentric human video and internet video, because both scale coverage across tasks and environments without needing the physical robot. Mid-training and post-training align that prior to the target embodiment and lean more on teleoperation, or on egocentric data paired against the specific robot through a retargeting step, because the policy is now being graded on the exact hardware it will run on.

Most humanoid programs blend sources rather than choosing one: a broad pretraining mixture drawing on egocentric human video, internet video, and simulation for coverage, then a smaller, targeted mid-training and post-training stage on teleoperated or egocentric data aligned to the exact robot. Egocentric human data pairs with teleoperation data specifically because it scales coverage cheaply while teleoperation grounds the result in the exact robot embodiment — neither replaces the other outright.

Firsthand captures the egocentric human side of this mixture — head and wrist camera video, depth, 3D hand and body pose, and action segments — to a written spec. It does not capture teleoperation or robot-embodiment data; that side of the mixture comes from a program’s own robot-and-operator pipeline, aligned against the egocentric corpus.

A written skill spec covering tasks, environments, and accept criteria is what turns any of these methods into usable coverage, whichever source is doing the collecting. The spec is what a provider or an internal team validates delivered footage against, batch by batch.