Glossary
What Is Pretraining in Robot Learning?
Last reviewed: September 2026
Short answer
Pretraining is the first stage of training a robot foundation model: a policy learns broad, general representations from a large corpus of egocentric human video, robot demonstrations, or both, before being adapted to any specific task. The goal is coverage and scale, not task performance — mid-training and fine-tuning (post-training) come after to align and specialize the model.
01
Why it matters
A robot foundation model’s pretraining breadth sets a ceiling on what it can generalize to later. NVIDIA’s EgoScale reports a log-linear scaling law between pretraining data volume and downstream task performance, and Generalist’s GEN-0 frames quality and diversity as mattering as much as sheer hours — so pretraining data planning is a scale-and-diversity problem, not just a volume problem. Published pretraining corpora vary by roughly two orders of magnitude in scale: EgoVerse reports 1,362 hours, Egocentric-10K reports roughly 10,000 hours, EgoScale reports 20,854 hours, and GEN-0 reports 270,000+ hours — the differences reflect different goals and budgets, not one agreed-upon minimum.
02
How Firsthand handles it
Firsthand captures the synchronized egocentric video, depth, 3D hand pose, and action-segment data used for pretraining corpora, collected fast, on demand, and to a written spec, then delivered in RLDS, LeRobot format, or WebDataset with a buyer-owned commercial license. Because pretraining data is used before any task-specific adaptation, the spec that matters most is coverage — scenes, tasks, objects, and people — rather than any single volume target, and a custom collection can be scoped around exactly the coverage gaps a public corpus does not fill.
03
Pretraining vs mid-training vs post-training, at a glance
| Stage | Goal | Typical data |
|---|---|---|
| Pretraining | Broad coverage and scale across tasks, scenes, and (often) embodiments | Large egocentric human-video or robot-demonstration corpora, e.g. EgoScale’s 20,854-hour pretraining set |
| Mid-training | Align human and robot embodiments on matched tasks and viewpoints | A smaller, tightly aligned human-plus-robot slice, e.g. EgoScale’s ≈50h human + ≈4h robot across 344 tabletop tasks |
| Post-training (fine-tuning) | Specialize the model to a specific deployment task, robot, or environment | A written skill spec collected against the target task and edge cases |
05
Published pretraining corpora, by scale
| Corpus | Reported scale | Source |
|---|---|---|
| EgoVerse | 1,362 hours | arXiv 2604.07607 |
| Egocentric-10K | ~10,000 hours | Hugging Face (builddotai/Egocentric-10K) |
| EgoScale | 20,854 hours (pretraining) | arXiv 2602.16710 / NVIDIA GEAR |
| GEN-0 | 270,000+ hours | Generalist AI official blog |
06
Frequently asked questions
- What is pretraining in robot learning?
- Pretraining is the first stage of training a robot foundation model, where a policy learns broad, general representations from a large corpus of egocentric human video, robot demonstrations, or both, before being adapted to any specific task.
- What data is used for robot pretraining?
- Large-scale egocentric human video and robot demonstration data. Published examples include EgoScale’s 20,854-hour corpus, Generalist’s GEN-0 (270,000+ hours), NVIDIA’s GR00T N1 data pyramid, and the public EgoVerse and Egocentric-10K datasets — see how much pretraining data does a robot need for the full comparison.
- How does pretraining differ from mid-training and fine-tuning?
- Pretraining maximizes breadth and scale. Mid-training is a smaller, tightly aligned stage — EgoScale used it to align human and robot embodiments on matched tasks. Fine-tuning (post-training) specializes the model to a specific deployment task, robot, or environment. See pretraining vs mid-training vs post-training data for the full breakdown.
- Why does pretraining matter for robot foundation models?
- Pretraining breadth sets a ceiling on downstream generalization. EgoScale reports a log-linear scaling law between pretraining data volume and task performance, and GEN-0 frames quality and diversity as mattering as much as raw hours — so pretraining coverage, not just volume, determines what a model can later be adapted to.
- Does more pretraining data always mean a better model?
- Not by itself. See do robot foundation models follow data scaling laws for how EgoScale and GEN-0 both frame data composition and diversity as factors alongside raw hours, rather than hours alone.
- Can pretraining data come from human video instead of robot demonstrations?
- Yes. EgoScale’s pretraining corpus is egocentric human video, and EgoScale later showed adding a much smaller aligned slice of human-plus-robot mid-training data can outperform larger robot-only baselines. See human video vs robot data for pretraining for the comparison.
- How much pretraining data is typical?
- Published corpora range widely: EgoVerse reports 1,362 hours, Egocentric-10K reports roughly 10,000 hours, EgoScale reports 20,854 hours, and GEN-0 reports 270,000+ hours. See how much pretraining data does a robot need for the full comparison and what it implies for planning.
- Is pretraining data the same thing as a data pyramid?
- No, but they are related. NVIDIA’s GR00T N1 describes a data pyramid — layered by scale and specificity, with the broadest layer at the base. Pretraining data typically populates that base layer, while mid-training and fine-tuning data occupy the smaller, more specific layers above it. See building a robot pretraining data mixture for how the layers combine.