Human Video vs Robot Data for Pretraining

For pretraining specifically, the evidence on this site points toward egocentric human video: it scales to far more hours, scenes and tasks per collection hour than a robot-and-operator fleet can reach, and EgoScale’s log-linear scaling law shows that scale keeps improving downstream performance. Robot data, teleoperated or otherwise, is not absent from the pipeline — it typically does the mid-training and post-training work of aligning and fine-tuning that pretrained prior to one target embodiment.

The two sources are not competing for the same job. See /guides/robot-pretraining-data for how pretraining, mid-training and post-training divide the work across a full training pipeline.

Egocentric human video records a person performing a task directly, typically on a worn camera rig at the head and wrists. Teleoperated robot data records a human operator driving the physical robot itself — through direct teleoperation, VR or motion-capture teleoperation, or a leader-follower rig — so the robot’s own joints and end-effector generate the action data.

The practical difference is embodiment. Egocentric human video captures a human hand and body, which has different kinematics and degrees of freedom than a robot end-effector — closing that gap needs a retargeting or alignment step, covered further in /compare/human-demonstrations-vs-teleoperation. Teleoperated robot data has no such gap, because it is already the target robot moving, but every hour of it requires the physical robot, a controller rig and an operator present at the same time.

HumanScale is a separate published research line specifically studying the scaling behavior of human-video pretraining for robot policies. Alongside EgoScale, it adds evidence that studying how human-video scale affects downstream robot performance is now an active, evidence-backed area of robotics research rather than the result of a single lab or a single paper.

EgoScale pretrained on 20,854 hours of action-labeled egocentric human video and found a log-linear scaling law between data scale and validation loss — more human video at pretraining scale kept improving the metric, and the paper reports that trend correlating with downstream real-robot task performance.

After that broad pretraining stage, EgoScale added a lightweight mid-training stage: roughly 50 hours of human video paired with about 4 hours of robot data across 344 tabletop tasks, aligned closely to one target embodiment. On a 22-degree-of-freedom dexterous hand, that recipe delivered a 54% average improvement in task success rate compared with training with no human pretraining at all. The finding is not "human video alone is enough" — it is that a large human-video pretraining stage plus a small, closely matched robot-aligned stage outperforms skipping the human-video stage. See the full breakdown at /compare/egoscale.

Egocentric human video scales faster and cheaper per hour because it only needs a worn rig and a participant — no robot hardware, controller rig or facility booking required for every session. That is why EgoScale’s pretraining corpus reaches 20,854 hours: a scale a robot-and-operator fleet would need far more calendar time and budget to match.

Teleoperated robot data is slower and more expensive per hour, bound by robot and operator availability, but it buys something human video cannot: zero embodiment gap. The action data is already expressed in the target robot’s own joint or end-effector commands, with no retargeting step required before a policy can consume it directly.

Even with a large human-video pretraining stage, EgoScale’s own recipe still needs robot-aligned data before deployment. Its mid-training stage — roughly 50 hours of human video plus about 4 hours of robot data across 344 tabletop tasks — narrows the broad pretrained prior to the specific embodiment it will run on. That mirrors the general pattern this site documents in /guides/robot-pretraining-data and /guides/pretraining-vs-post-training-robot-data: pretraining is broad and human-sourced, while mid-training and post-training (fine-tuning) are smaller datasets closely matched to the deployment embodiment and target skill set.

Post-training or fine-tuning data — whether teleoperated robot demonstrations or closely matched human demonstration paired with the target robot — is where the embodiment gap matters most, because the policy is now specializing rather than building a general prior.

The pattern that shows up across EgoScale and the general pretraining-to-post-training structure on this site is staged, not blended arbitrarily: a large human-video pretraining stage builds the broad prior, then a small, closely matched mid-training mixture (human plus a little robot data, as in EgoScale’s ~50h/~4h split across 344 tasks) aligns it to the target embodiment, then post-training fine-tunes on the deployment robot and skill set.

Firsthand delivers custom batches in RLDS, LeRobot, WebDataset, HDF5, zarr and Rerun .rrd, so custom egocentric human-video collections mix cleanly alongside existing teleoperation or robot data at whichever stage a program needs to fill. See /compare/human-demonstrations-vs-teleoperation for how human and teleoperated data specifically pair for embodiment alignment.

Describe the skill and environment diversity your pretraining corpus is missing. That becomes a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, quoted before capture begins — often starting with a Spec Pilot, a small first batch that proves the spec before scaling volume. See /custom-collection/pretraining-data-collection for the full engagement model.