Guide
Robotics Dataset Quality Checklist
Short answer
A robotics dataset is quality-checked, not quality-claimed. Before buying or using one, verify it reports a documented cross-stream sync error under a fixed ceiling, counts validated hours rather than raw hours, includes failure and near-miss cases, ships a per-participant consent chain, and delivers in a standard training format under a clear, buyer-usable license.
Last reviewed:
How should this checklist be used?
Each item below is a specific, checkable claim a provider should be able to answer directly — not a general quality assurance. Run through it against a sample pack or a provider’s documentation before committing to a full collection or dataset purchase. For the reasoning behind each signal, see the companion guide on what makes robotics training data high quality.
- Cross-stream sync error is documented
- A stated, measured number in milliseconds, with a fixed rejection ceiling — not a hardware spec assumption.
- Hours are reported as "validated," not raw
- Only footage that passed a validation check against the sync ceiling and capture spec counts toward the delivered total.
- Failure and near-miss cases are included
- The dataset includes stalls, slips, and corrections, not only clean successful demonstrations.
- A per-participant consent chain ships with the data
- Documented consent tied to the specific footage each participant appears in, traceable clip by clip.
- Delivery format matches your training stack
- Standard formats such as RLDS, LeRobot, or WebDataset, not a proprietary format that requires a custom loader.
- The license is clear and buyer-usable
- A stated license that covers commercial model training, not an ambiguous or research-only grant.
Six pass/fail checks. If a provider cannot answer one directly, treat that as a gap, not an assumption in your favor.
Why a checklist instead of a single quality score?
A single "quality score" hides which specific signal is weak. A dataset can have excellent sync but no failure-case coverage, or good coverage but no documented consent chain — and each of those gaps affects a downstream policy differently. Checking each signal independently makes it clear exactly what you are and are not getting.
What if a dataset fails one of these checks?
A failed item does not necessarily rule a dataset out — it depends on what you are training. A perception-only model may not need failure-case coverage; a manipulation policy usually does. The point of the checklist is to make that trade-off visible up front, rather than discovering the gap after training. For a custom collection, each of these six items is defined directly in the capture spec rather than inspected after delivery.
FAQ
Robotics Dataset Quality Checklist, answered.
01What is the fastest way to check dataset quality before buying?
Ask for the sync-error number and ceiling, the validated-hour count (not raw hours), whether failure cases are included, and the consent-chain documentation. Those four answers surface most quality gaps quickly.
02Is a low sync-error number always necessary?
It matters most for tasks where visual, depth, and pose streams need to agree on timing, such as contact-rich manipulation. A perception-only classification task is less sensitive to it, but the number should still be documented either way.
03Why does failure-case coverage matter for a checklist item?
A dataset built only from clean successes doesn’t show a model what a recoverable mistake or near-miss looks like, which matters for any policy expected to operate in the real world rather than a scripted demo.
04What counts as an acceptable consent chain?
A documented, per-participant record tied to the specific footage that participant appears in, so a buyer can trace exactly what was agreed to for any given clip — not a blanket statement that "consent was obtained."
05Does this checklist apply to public datasets too?
Yes. The same six checks apply whether the data comes from a public dataset or a custom collection — the difference is whether you can verify the answers directly or have to take the provider’s documentation at face value.
Check it against the sample pack.
40 episodes across 4 environments, delivered in the exact schema these guides describe.