Robotics Training Data RFP Checklist for AI Labs

A robotics training data RFP is a written request that asks one or more data providers to propose how they would capture, label, and deliver a specific dataset — rather than asking generically what a vendor offers. It names the task, the required capture setup, the QA bar, and the delivery terms up front, so every vendor is responding to the same requirements and their answers can be compared line by line.

An RFP for training data differs from a general procurement RFP in one important way: the deliverable is not a fixed off-the-shelf product but newly captured footage against a spec that does not exist yet. That makes the RFP itself the first draft of the skill spec the collection will actually run against, which is why it needs to be specific rather than open-ended.

An RFP fits when a lab has identified a specific gap — a task, embodiment, or condition mix an existing model or dataset does not cover — and needs to compare more than one vendor on cost, timeline, and capability before committing. It is not required for every data need: a lab that already has a trusted process pilot with a known provider, or a well-served task and population, can often move straight to a focused brief without a formal RFP.

The clearest signal that an RFP is worth the overhead is scale or novelty: a large production volume, a new embodiment, or a condition mix the lab has not sourced before all raise the cost of picking the wrong vendor, which is exactly what a comparable, scored RFP process is meant to reduce.

The RFP should name the task by verb and noun rather than by category — "load a dishwasher rack" rather than "kitchen manipulation" — because a vendor cannot price, staff, or schedule against an ambiguous task. It should also state the geometry that must stay in frame, the environments and condition cells the task must be captured across, and the target number of hours or episodes per cell, not just an aggregate total.

Asking vendors to respond with their own draft skill spec — not just a price — is one of the most useful things an RFP can require, because it reveals whether a vendor actually understands the task before any footage is captured. A vendor that returns a generic capability list instead of a spec proposal has not engaged with the specific task named in the RFP.

The RFP should specify what sensor streams the task requires — RGB, depth, IMU, audio, gaze, or robot-embodiment kinematics — and the calibration and synchronization tolerance those streams must meet. Cross-stream timing matters most for tasks involving fast contact or fine manipulation, where a few milliseconds of skew between vision and proprioception changes what a policy learns from the footage.

Rather than asking a vendor to describe their rig in the abstract, the RFP should ask for the actual measured sync residual on delivered footage — a median and a tail figure, not a marketing claim — and how often calibration is checked during a capture session rather than assumed from a one-time setup.

The RFP should name the required annotations directly — action segments, object masks, 3D hand pose, language instructions, or task success labels — and the delivery format each one must ship in, since a label that cannot be loaded into the training pipeline in its delivered format is not usable regardless of how accurate it is.

It should also ask how labels are validated, not just produced: whether annotation passes through independent review, what the disagreement or correction rate on a sample batch looks like, and whether that rate is reported per batch rather than only claimed once at the start of the engagement.

The RFP should require a written, testable accept and reject rule before any footage is captured, not after delivery. Without a stated rule, "done" is undefined and disagreements about quality surface only once a batch has already shipped, which is far more expensive to resolve than agreeing on the rule up front.

It should also require the billing unit to be a validated hour — footage that has already cleared the accept criteria — rather than a raw capture hour before anyone has checked it, and require a reject log with reasons, not just pass counts, delivered with every batch so the lab can audit the bar the vendor is actually holding.

The RFP should ask whether every contributor signs a release before capture, and — critically — whether the same consent standard covers anyone who appears in frame incidentally, such as family members, coworkers, or bystanders, not only the person performing the task under contract.

It should also ask how personal data is removed from footage: whether faces, license plates, and screens are detected and blurred, whether that removal is irreversible, and whether a human reviewer confirms the result rather than trusting automated detection alone. Consent and anonymization answer two different questions — whether a person agreed to be recorded, and whether anyone who did not consent is still identifiable — and a strong RFP response addresses both separately.

The RFP should state explicitly that the delivered data must come under a perpetual, buyer-owned commercial license, not a shared, research-only, or revocable license. Many existing public and academic datasets ship under research-only terms such as CC BY-NC, which blocks commercial training and deployment without a separate license negotiation — an RFP for custom data exists precisely to avoid that constraint.

Ownership terms matter most at the point a trained model ships in a product: a lab that discovers only then that its training data carries a shared or restricted license faces a costly renegotiation or a retraining effort. Requiring the license terms in writing at the RFP stage removes that risk before any capture begins.

The RFP should name the delivery format the training pipeline actually expects — RLDS, LeRobot, WebDataset, HDF5, zarr, Rerun, WAV, JSONL, or another named schema — rather than leaving the format to be decided after capture starts, since a mismatched format means re-wrangling every batch before it is usable.

It should also require a data card with every delivery, documenting provenance, jurisdictions, and the consent and anonymization records behind the batch, plus a reject log showing what was rejected and why. Requesting a rolling batch cadence rather than one delivery at the end lets the lab start integrating data into training while the rest of the volume is still being captured.

The RFP should ask each vendor to propose a small pilot batch captured against the same draft spec the full engagement would use, rather than a free sample of footage that already exists. A pilot that is not captured against the real spec does not prove anything about how the full production run will perform.

The RFP should also state that the pilot and the production run share one spec, one accept/reject rule, and one billing unit — so a vendor cannot pass an easy pilot and then quietly loosen its criteria once full volume is committed. Requiring the pilot to report a reject log and per-condition-cell fill rate, not just an aggregate pass rate, is what makes it possible to check that promise.

See scaling a robotics data pilot to production for the full checklist of what a pilot should prove before a lab commits to full volume.

Scoring vendor responses against a written scorecard, rather than a general impression, keeps the comparison objective across proposals that differ in format and detail. Each RFP requirement above becomes a scoring row: a vendor either answers it with specifics, answers it vaguely, or does not answer it at all.