How to Choose a Robot Pretraining Data Provider

A robot pretraining data provider is being asked to deliver a very different product than a small, targeted fine-tuning batch. Pretraining-scale data has to span far more scenes, tasks, objects and people, carry consistent action labels and calibration across the whole corpus, and be delivered under a license that supports training a commercial model — not just a research checkpoint. The right way to evaluate a candidate is the same due-diligence process used for any robotics data vendor: write a spec, request a sample against it, score the sample on a weighted scorecard, and only then commit to volume.

This page focuses the general vendor-evaluation process on the specific questions that matter for pretraining-scale corpora. For the broader due-diligence process and scorecard mechanics, see how to evaluate a robotics data vendor; for the underlying evaluation criteria in more depth, see how to choose an egocentric data provider.

It also helps to be clear about what stage of training the corpus is for before you evaluate a vendor at all. Pretraining data sits at the base of a foundation model’s training and is deliberately broad and general; mid-training data narrows that base toward an aligned human-plus-robot slice; post-training or fine-tuning data narrows further to one target robot and one task. A vendor that is strong at fine-tuning-scale collection for a single skill does not automatically have the operational capacity, coverage planning, or consistent calibration pipeline that pretraining-scale volume requires, so confirm which stage of the pyramid a candidate has actually delivered at before assuming their process transfers.

Pretraining corpora need breadth across four axes at once: scenes, tasks, objects and people. A vendor that can produce a large hour count in one kitchen with one operator is not the same as one that can produce hours spread across many environments, task types, object categories and demonstrators. Ask for a coverage histogram broken out by all four axes, not a single aggregate hour total, so thin cells in the distribution are visible before you buy.

A provider that publishes fill rates and gaps by condition is telling you where the data is thin before you commit to volume — the same signal the general vendor scorecard weights highly for any robotics data purchase, applied here at pretraining scale.

Diversity planning also needs to hold up across an entire pretraining run, not just the first delivered batch. Ask how the provider maintains the same diversity ratios as volume scales from a pilot to a full corpus: a vendor that front-loads its most convenient scenes and demonstrators into an early sample, then fills the remainder of the order with whatever is fastest to capture, quietly narrows the diversity that made the sample look strong in the first place. A written diversity plan, checked against delivered footage batch by batch rather than only at the end of the order, is the way to catch that drift before it reaches your training run.

Ask exactly what action representation ships with the corpus: 3D hand pose, metric depth, and frame-accurate action segments are the manipulation-ready baseline; RGB and narrations alone are not. Ask how camera intrinsics and extrinsics are calibrated and whether that calibration is documented consistently across the whole corpus, not just a handful of sample clips — a pretraining corpus with inconsistent calibration across capture sessions produces action labels that are not directly comparable batch to batch.

Ask how the vendor bills and measures quality. Billing per validated hour aligns their incentive with your outcome; billing per raw hour pays for footage you may not be able to use. Ask for the measured sync error and the method used to produce it — a hardware-triggered sub-millisecond figure with drift written to metadata is a different claim than "software-aligned." Ask whether the corpus includes failure cases and recoveries, not only clean successful demonstrations, since a policy that never sees a failure in training data has no signal for what a recovery looks like.

Ask for a reject policy and, ideally, a reject log: a vendor that discards off-spec footage and shows you what it discarded is protecting your training signal rather than maximizing delivered hours.

Ask, too, how quality is measured for the parts of the corpus that carry no ground-truth action label at all. Some pretraining recipes deliberately include large volumes of unlabeled human video and infer action information after the fact, through latent-action learning or inverse-dynamics pseudo-labeling, rather than requiring every clip to arrive with a hand-annotated label. If a candidate provider is delivering that kind of unlabeled-plus-inferred mix, ask what inference method they use and how its accuracy is checked, since a bad pseudo-label pipeline can silently degrade a large share of a pretraining corpus in a way that a simple hour count would never reveal.

Confirm the license explicitly permits training and shipping a commercial model, and confirm who owns the resulting data and derived models. A research-only or non-commercial license can silently block a commercial product later, even if the data trains a working policy today. Ask for a written per-participant consent release, delivered as an artefact per batch rather than promised verbally, and ask how personal identifying details are anonymized in delivered footage.

This page describes buyer-side evaluation criteria, not legal advice. Confirm license and consent terms directly with any provider and, where needed, with your own counsel before training a commercial model on licensed data.

Confirm the delivery format matches what your training pipeline actually consumes. Training-ready pretraining corpora are typically delivered in RLDS, LeRobot, WebDataset, or HDF5. Ask for a sample in the exact format you will train on, not a format that requires a conversion step you have not tested, since format conversion at pretraining scale can introduce its own bugs in action labels or timestamps.

Also ask how metadata and labels are packaged inside the format, not just which format name is on the delivery. Two vendors can both say "RLDS" while one embeds full calibration, sync-error metadata and per-clip consent references in every shard and the other ships a bare action trajectory with the supporting documentation held separately in a spreadsheet. At pretraining scale, where a single order can span thousands of individual capture sessions, metadata that lives outside the data files is metadata that is easy to lose track of during ingestion — ask for a sample shard and check what is actually embedded before you commit to volume.

Do not commit to full pretraining-scale volume on the strength of a demo reel. Write a skill spec that names the scenes, tasks, objects and people you need coverage across, then request a small sample or pilot batch against that exact spec. Score what comes back on a weighted scorecard covering sync error, annotation depth, license, consent, reject policy and billing unit, and only scale to a full custom collection once the pilot clears every high-weight row.

A pilot is also the point at which you find out whether a vendor’s process is reproducible at all. Ask for two samples captured at different times, ideally by different operators, against the same spec, and check whether calibration, sync error and label quality hold steady between them. A single strong sample tells you a vendor can produce good data once; a pair of consistent samples tells you the process itself is under control, which is the property that matters once an order scales from a pilot batch to a full pretraining-scale corpus spread across many capture sessions and, often, many collection sites.