What Is Co-Training in Robot Learning?

A robot policy that only ever sees its own robot’s teleoperated trajectories is limited to whatever that robot has physically demonstrated. Co-training lets a model see human video alongside robot data in the same batch, so the two sources can shape the same set of weights together instead of one being thrown away once the next stage starts. EgoVerse’s paper (arXiv 2604.07607) reports that "co-training robot policies with human data leads to clear and reproducible performance improvements," validated "under a standardized, cross-lab experimental setup spanning multiple robots" — and that this benefit depends on how closely the human data aligns with the target robot’s task semantics, not on co-training alone. This is why co-training is discussed alongside mid-training: EgoScale frames its own aligned human-robot data as a co-training step inside its mid-training stage, rather than as a separate pretraining-then-fine-tuning sequence.

Firsthand does not train or co-train any model. It captures the egocentric human side of a co-training mixture — video, depth, 3D hand pose, and action-segment labels, with episode fields for scene, task, and object metadata — collected fast, on demand, and to a written spec, then delivered in RLDS, LeRobot format, or WebDataset with a buyer-owned commercial license. Because the published findings tie co-training benefit to alignment between the human data and a target robot’s own task semantics, a program planning to co-train can scope a custom collection around matching those semantics — similar tasks, similar scenes, similar viewpoints — rather than collecting egocentric video independently of what the robot side already covers.

What does co-training mean in robot learning?

Co-training means training one policy on more than one data source at the same time — for example, egocentric human video and robot demonstrations in the same training run — rather than training on each source in a separate, sequential stage.

Why do robot models co-train on human and robot data?

Because robot-only data is expensive and limited in scale, while human video is comparatively abundant. GR00T N1’s paper describes co-training across its data pyramid in both pre- and post-training, and EgoScale introduces its aligned human-robot mid-training data through co-training, so the model can draw on both sources within the same training process.

What do published studies say about co-training?

GR00T N1’s paper (arXiv 2503.14734) states it develops "an effective co-training strategy to learn across the entire data pyramid in both pre- and post-training phases." EgoScale’s paper (arXiv 2602.16710) introduces aligned human-robot mid-training data "through co-training." EgoVerse’s paper (arXiv 2604.07607) reports that co-training robot policies with human data produced clear, reproducible performance improvements across a multi-lab, multi-robot study, and that the benefit depends on alignment between the human data and the target robot’s task semantics.

How is co-training different from mid-training and fine-tuning?

Mid-training and fine-tuning (post-training) are sequential stages — one dataset is used, then training moves to the next, more specialized dataset. Co-training is a way of combining sources within a stage: EgoScale, for instance, uses co-training inside its mid-training stage to combine aligned human and robot data in the same training process rather than training on each in turn.

How is co-training different from a data mixture?

A data mixture is the composition — which sources and in what proportions — that goes into a training run. Co-training is the training-time practice of actually combining those sources in the same run or batch, rather than training on them in separate stages. A data mixture describes what is in the batch; co-training describes how it is trained on.

What does co-training mean for capturing egocentric data?

Published findings tie co-training benefit to how closely the human data’s task semantics align with the target robot’s own tasks. That means a custom egocentric collection intended for co-training should be scoped around matching scenes, tasks, and viewpoints to what the robot side already covers — captured with hand pose, action-segment labels, and episode metadata so the two sources stay comparable — rather than collected independently of the robot data it will be trained alongside.

Does Firsthand co-train any models?

No. Firsthand captures the egocentric human video, depth, hand pose, and action-segment data that a program can use in its own co-training run. Firsthand does not train, co-train, or own any model, and does not capture robot teleoperation data itself.

Is co-training the same as cross-embodiment learning?

They overlap but are not identical. Cross-embodiment learning describes training one policy on data from multiple embodiments so skills transfer across bodies. Co-training is the training-time mechanism that can make that possible — seeing multiple sources together in the same run — but co-training can also combine sources within a single embodiment’s own data at different stages.