EgoScale pretraining data

EgoScale is a research program from NVIDIA that studies how pretraining a robot foundation model on large-scale egocentric human video affects downstream manipulation performance. Its corpus and scaling results are documented in the EgoScale paper and on NVIDIA’s GEAR lab research page. For a shorter, plain-language explainer, see /guides/what-is-nvidia-egoscale.

EgoScale pretrained on 20,854 hours of action-labeled egocentric human video. The action labels were produced from off-the-shelf SLAM (for camera trajectory and scene geometry) and hand-pose estimation, rather than captured directly from a purpose-built synchronized rig.

EgoScale reports a log-linear scaling law between the amount of human pretraining data and validation loss — as human video scale increases log-linearly, validation loss keeps falling, and the paper reports this trend correlates with real-robot task performance downstream rather than being purely an offline metric.

EgoScale’s recipe has two stages. First, large-scale pretraining on the full 20,854-hour human video corpus builds a broad visuomotor prior. Second, a lightweight, aligned human-robot mid-training stage narrows that prior to a target embodiment, using roughly 50 hours of human video and about 4 hours of robot data across 344 tabletop tasks.

The pattern — broad human pretraining first, then a small aligned mid-training dataset — mirrors the general training-stage structure this site documents in /guides/robot-pretraining-data: pretraining is broad and human-sourced, mid-training is a smaller dataset closely matched to the deployment embodiment.

EgoScale’s labels come from off-the-shelf SLAM and hand-pose estimation applied to the raw egocentric video, not from a purpose-built capture rig with hardware-synchronized sensors. That approach scales to tens of thousands of hours, but it means the labels are estimated after capture rather than measured directly on a rig with a shared hardware clock across RGB, depth and pose.

On a 22-degree-of-freedom dexterous hand, EgoScale reports a 54% average improvement in task success rate compared with training with no human pretraining at all — the clearest downstream evidence in the paper that the pretraining-plus-mid-training recipe transfers to real manipulation performance.

EgoScale is a research paper and dataset release; whether a specific release’s terms permit commercial training is a question to confirm directly against its published license before relying on it for a commercial model. That is the general pattern with research-scale egocentric corpora: many carry research-only or non-commercial terms, which is why a commercial pretraining program typically pairs a public-scale reference point like EgoScale with a custom collection licensed explicitly for commercial training from the outset.

EgoScale is strong evidence that scale in egocentric human video keeps helping pretraining, and that a small, aligned mid-training stage is enough to transfer that prior to a real robot. It does not settle what a corpus needs beyond hours: EgoScale’s own labels are estimated (SLAM plus hand-pose estimation) rather than captured on a synchronized rig, and its license status has to be confirmed per release.

See /guides/robot-pretraining-data for how pretraining, mid-training and post-training fit together across the field, /guides/robot-data-scaling-laws for what EgoScale’s scaling law does and doesn’t tell you, and /custom-collection/pretraining-data-collection for what a custom, commercially licensed pretraining collection captures instead of estimating after the fact.

Describe the skill and environment diversity your pretraining corpus is missing. That becomes a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, quoted before capture begins — often starting with a Spec Pilot, a small first batch that proves the spec before scaling volume. See /custom-collection/pretraining-data-collection for the full engagement model.