Glossary
What Is Mid-Training?
Last reviewed: September 2026
Short answer
Mid-training is the middle stage of a three-stage robot foundation model training recipe: after broad pretraining on human video or mixed data, a smaller, tightly aligned human-plus-robot dataset adapts the pretrained representation to one target robot’s embodiment — matched viewpoint, calibration, and tasks — before task-specific fine-tuning (post-training) narrows further to a deployment task.
01
Why it matters
A pretrained representation is broad but not aligned to any one robot. NVIDIA’s EgoScale showed that a much smaller, tightly aligned slice of human-plus-robot mid-training data — roughly 50 hours of human video paired with about 4 hours of robot data across 344 tabletop tasks — can outperform larger robot-only baselines. That result reframes mid-training as an embodiment-alignment problem, not a volume problem: the value comes from how closely the mid-training set’s camera geometry, calibration, and tasks match the target robot, not from adding more hours.
02
How Firsthand handles it
Firsthand captures matched human-and-robot mid-training data — synchronized on a single hardware clock, with calibrated intrinsics and hand pose or action-segment labels — collected fast, on demand, and to a written spec, then delivered in RLDS, LeRobot format, or WebDataset with a buyer-owned commercial license. Because mid-training data is measured against one specific robot, the spec centers on matching that robot’s viewpoint and reach rather than maximizing hours. A mid-training program typically records a human contributor and the target robot performing the same or closely related tasks on the same capture and calibration setup, so the two streams stay directly comparable frame for frame.
03
Where mid-training sits between pretraining and post-training
| Stage | Goal | Typical data |
|---|---|---|
| Pretraining | Broad coverage and scale across tasks, scenes, and (often) embodiments | Large egocentric human-video or robot-demonstration corpora, e.g. EgoScale’s 20,854-hour pretraining set |
| Mid-training | Align human and robot embodiments on matched tasks and viewpoints | A smaller, tightly aligned human-plus-robot slice, e.g. EgoScale’s ≈50h human + ≈4h robot across 344 tabletop tasks |
| Post-training (fine-tuning) | Specialize the model to a specific deployment task, robot, or environment | A written skill spec collected against the target task and edge cases |
05
What a mid-training capture setup needs
| Requirement | Why it matters |
|---|---|
| Matched viewpoint | Camera geometry has to resemble the target robot’s cameras, or the alignment benefit does not transfer |
| Calibrated intrinsics | EgoScale’s mid-training stage depends on knowing exact camera parameters for both human and robot streams |
| Single hardware clock sync | Human and robot episodes must be comparable frame-for-frame across the same capture pipeline |
| Paired human + robot capture | The same or closely related tasks recorded on both a human contributor and the target robot |
06
Frequently asked questions
- What is mid-training in robot learning?
- Mid-training is the middle stage between broad pretraining and task-specific fine-tuning: a smaller, tightly aligned human-plus-robot dataset that adapts a pretrained representation to one target robot’s embodiment before fine-tuning narrows to a specific deployment task.
- Where does mid-training sit between pretraining and post-training?
- Pretraining is broad and embodiment-agnostic, collected across many skills and environments with no single robot in mind. Mid-training narrows that to one target robot’s camera geometry and reach. Post-training (fine-tuning) narrows further still, to a specific deployment task. See pretraining vs mid-training vs post-training data for the full breakdown.
- What data is used for mid-training?
- A smaller, closely matched human-and-robot dataset. EgoScale’s published recipe pairs roughly 50 hours of human video with about 4 hours of robot data across 344 tabletop tasks — dozens of hours, not thousands, because the goal is embodiment alignment rather than scale.
- How does mid-training help close the embodiment gap?
- By matching viewpoint, calibration, and tasks between human and robot capture rather than by adding more hours. EgoScale showed this smaller, aligned mid-training slice can outperform larger robot-only baselines, because the alignment — not the volume — is what transfers a pretrained representation onto a specific robot.
- What capture requirements does aligned human-robot mid-training data have?
- Matched camera viewpoints between the human contributor and the target robot, calibrated intrinsics for both, a single hardware clock synchronizing every stream, and hand pose or action-segment labels so the two capture types stay directly comparable.
- Is mid-training the same as fine-tuning?
- No. Mid-training adapts a pretrained representation to one robot’s embodiment using matched human-and-robot data. Fine-tuning (post-training) comes after and specializes the model to a specific deployment task, robot, or environment using a written skill spec.
- Do all robot foundation models use a mid-training stage?
- Not necessarily — some training recipes go directly from pretraining to fine-tuning. EgoScale is the published example of a recipe that explicitly inserts an aligned mid-training stage between the two, and reports it improves on robot-only baselines.
- How much mid-training data is typical?
- EgoScale’s published recipe used roughly 50 hours of human video and about 4 hours of robot data across 344 tabletop tasks — far smaller than its 20,854-hour pretraining corpus, because mid-training scale requirements are driven by alignment coverage, not raw volume.