Aligned human-robot mid-training data.

Mid-training data is the second stage in a three-stage training recipe for robot foundation models: broad pretraining on human data first, then a smaller, aligned human-robot mid-training stage that adapts the pretrained representation to a target embodiment, before task-specific fine-tuning on the deployment robot.

It sits between pretraining and post-training in both scale and specificity. Pretraining is collected broadly, across many skills and environments, with no single robot in mind. Mid-training is collected with one target robot’s camera geometry and reach already fixed, so every episode is directly comparable to what that robot will see and do.

EgoScale documents a two-stage recipe: large-scale pretraining on a 20,854-hour human egocentric corpus, followed by a lightweight, aligned human-robot mid-training stage. That mid-training stage pairs roughly 50 hours of human video with about 4 hours of robot data, collected across 344 tabletop tasks (arXiv 2602.16710; NVIDIA GEAR Lab project page).

The 54% success improvement EgoScale reports on a 22-DoF hand comes from the full two-stage recipe, not from pretraining alone. The pretraining corpus establishes the broad prior; the aligned mid-training stage is what carries that prior onto the specific hand it was evaluated on.

A model trained on broad human pretraining data has learned general priors about how hands and objects move, but it has not learned the specific camera geometry, field of view, or reach of any one robot. Mid-training closes that gap by recording from a viewpoint and calibration that match the target robot as closely as possible.

Calibrated intrinsics matter because mid-training data is measured against a specific embodiment: if the camera’s field of view, lens distortion, or mounting offset differs from the deployment robot’s, the alignment the stage is meant to provide is weakened. The same hardware-synchronized clock and calibrated-camera approach used at pretraining scale applies here, just against a narrower target.

EgoScale’s published recipe uses roughly 50 hours of human video and about 4 hours of robot data — dozens of hours, not thousands. That is a small fraction of its 20,854-hour pretraining corpus, and the gap between the two stages is deliberate: a pretraining corpus needs breadth across many tasks and settings, while a mid-training set needs precision on one embodiment.

The ratio is the useful takeaway for planning a program: pretraining scale is measured in tens of thousands of hours; mid-training scale is measured in dozens. What mid-training needs is not more volume but tighter alignment — matched viewpoint, calibrated intrinsics, and coverage of the specific tasks the target robot will perform.

Pretraining is broad, human-sourced, and collected without a specific robot in mind — its goal is a general visual and manipulation prior. Mid-training is smaller and embodiment-specific, aligning that prior to one robot’s camera geometry and reach using matched human and robot data. Post-training (fine-tuning) is the smallest and most specific stage, scoped to the exact deployment task on the exact deployment robot.

Each stage narrows scope and raises cost per hour as it goes: pretraining spans many skills and settings at the lowest cost per hour; mid-training narrows to one embodiment; fine-tuning narrows further to one task. A program that skips mid-training and jumps from broad pretraining straight to task-specific fine-tuning loses the alignment step that EgoScale’s recipe shows carries the pretrained prior onto a specific robot.

A mid-training collection pairs human contributor capture with target-robot capture on the same calibrated rig, so both streams share the same camera intrinsics and a single hardware clock. Hand pose, contact events, and action-segment labels are captured live rather than estimated afterward, matching the manipulation-grade signal used at pretraining scale.

Coverage is set by a written spec against the target robot’s task list — the specific manipulation or locomotion behaviors it needs to align to — rather than collected as a generic pile of tabletop video. Delivery follows the same formats used across the rest of the pipeline: RLDS, LeRobot, WebDataset, or HDF5, under a perpetual, buyer-owned license.

Because mid-training and pretraining run on the same underlying capture pipeline, a program can move from broad pretraining coverage to a closely aligned mid-training batch without switching data providers or formats. If a team already has some robot teleoperation data from its own fleet, a mid-training collection can be scoped to add matched human video against the same task list, filling the human side of the pairing EgoScale’s recipe describes.

The key planning question is which side is missing: teams with robot data but no matched human video need the human-capture half of the pairing; teams with broad human pretraining data but no embodiment-specific alignment need the robot-matched half. A written spec can scope either direction, or both, against one target robot.

The most common planning mistake is treating pretraining and fine-tuning as the only two stages that matter, and sizing a program as if broad human video plus a handful of task-specific robot demonstrations is enough on its own. EgoScale’s recipe shows the aligned mid-training step in between is what actually carries a broad prior onto a specific robot; without it, a fine-tuning set has to do double duty — both aligning to the embodiment and mastering the task — with far fewer hours than either job needs on its own.

The second common mistake is sizing a mid-training request the same way a pretraining request is sized: by hour count alone. Because mid-training’s value comes from viewpoint match and calibration precision rather than volume, a spec that only specifies a target number of hours — without naming the target robot’s camera geometry, reach, and task list — risks producing a batch that is the right size but the wrong shape for the embodiment it is meant to align to.

A mid-training spec differs from a pretraining spec in what it has to name up front. A pretraining spec can stay broad — general household or workplace coverage, a wide task and object mix — because the goal is a general prior, not alignment to one robot. A mid-training spec has to name the target robot’s specific camera mounting position, field of view, and reach, because every episode is measured against how closely it matches that geometry.

It also has to name the task list the mid-training stage is meant to align to, since coverage is scoped to the target robot’s specific behaviors rather than collected as generic tabletop video. And it should state which side of the human-robot pairing already exists — robot teleoperation data with no matched human video, human pretraining data with no embodiment-specific alignment, or neither — so the collection can be scoped to fill exactly the gap a program has, rather than duplicating data it already owns.