How to Scope a Custom Robotics Data Collection Project

Scoping is the work of turning a model problem into a written brief that a collection can be run, checked and billed against. It answers what will be captured, under which conditions, in what volume, with which labels, in which format and under which license. The written output is the skill spec; the scoping decisions on this page are what go into it.

You do not need a finished spec to begin. Firsthand drafts the spec with you on a working call, starting from the failure the model has today, and hands it back as a file you approve before anyone is briefed. A buyer who arrives with a demonstrated failure gives the spec a concrete target, which shortens the path to a signed spec.

This page covers the buyer-side decisions. For the stage-by-stage pipeline see how custom robotics data collection works; for the spec template see how to write a skill spec; for scoring vendor proposals see the robotics data RFP checklist.

Five decisions carry most of the scope. Each one maps to something that is either written into the spec or agreed during specification, so none of them is left to be discovered after delivery.

Name the exact task by verb and noun, not a category: "dice an onion", not "cooking". State the geometry that must stay in frame, such as both hands at grasp and the object throughout the episode. A vague verb lets capture drift toward whatever is convenient to film, and an unstated geometry lets the object of interest fall out of view.

Then budget conditions by cell rather than as one total. A policy fails on the conditions it never saw, so a budget that lists daylight, low light, clutter and failure cases separately shows where coverage is thin before training does. The same cells are what fill reports are measured against at every batch.

Also name the population the task needs: which languages, age bands, devices and environments. Recruitment draws on a global contributor pool screened against those cells, so the population named in the spec is the population captured. Leaving it implicit means finding the gap after delivery.

Scope volume in validated hours: footage that has already cleared the spec accept criteria, which is also the billing unit. Raw recording hours overstate what you can train on, because bulk footage is routinely discarded as unfit for manipulation.

As a starting point this site suggests 50–100 validated hours per skill, concentrated on the conditions the policy will actually meet, then scaling further only where held-out evaluation shows the policy is still thin. The guide on how much egocentric data you need explains the reasoning.

Volume interacts with the cell budget. Ask for hours where evaluation is weakest first, not an even spread, because the first hours per skill do the most work.

Raw capture is transcribed, labelled, keypointed, segmented or event-tagged as the spec requires, and structured into the delivery schema you name. Pick the annotation type from what the model needs: keypoints where joint or hand position matters, segmentation where an object boundary matters, event tags where sub-task timing matters, transcription where spoken content matters.

Name the target format before capture starts. Delivery ships in the format agreed during specification, including RLDS, LeRobot, WebDataset, HDF5, zarr, Rerun, WAV and JSONL, so the dataset loads into your existing pipeline without a bespoke parser or a reformatting pass.

If the model fuses several streams, say so in the scope. Every sensor shares one PTP (IEEE 1588) clock domain with hardware-triggered exposures, with a measured residual of a median 1.12 ms and a p99 of 1.94 ms against a 2.0 ms reject ceiling. An episode over the ceiling is discarded at capture, not shipped.

Agree the consent and licensing basis during specification, not after delivery. Every contributor signs a release before capture, and the same standard covers anyone who appears in frame incidentally. Faces, plates and screens are detected, tracked and blurred, with a human reviewer confirming the result.

Delivery is under a perpetual, buyer-owned commercial license. If the dataset must be collected only for you, state that in the scope: exclusivity is available at this stage rather than the same footage being resold to competitors.

Write the reject rule first and give one acceptance test a reviewer can apply, so a batch is accepted or rejected against a pre-agreed line rather than negotiated. Items that miss are rejected and recaptured at no extra cost, and the reject log, with reasons, is visible to you.

A spec pilot is optional. It is a small, fast first batch captured against the same spec, which surfaces the real reject rate, coverage fill and sync numbers before you commit to full volume. Teams with an unfamiliar condition mix, a rare embodiment or a spec they are not yet confident in get the most from one. A focused brief is also a normal starting point without one.

Discovery and specification come before the published speed numbers. After a spec is signed, first validated data arrives in a median of 48 hours where contributor coverage is deep, and in three to four weeks where coverage is thin and operators must be recruited and vetted. Scaling to full volume runs the same pipeline with the same accept rules.

Thin segments get a realistic staffing timeline agreed during specification rather than an optimistic one. Your own internal approvals sit outside these figures, so plan for them separately.

• The skill named by verb and noun, with the geometry that must stay in frame.
• A condition budget by cell, including failure cases, and the named population.
• A volume target in validated hours.
• The labels the data must carry and the delivery format and schema.
• The consent and license basis, and whether exclusivity is required.
• A reject rule, one acceptance test, and a decision on a spec pilot.

Firsthand does not state a price on this site, so cost is not covered here. Ask for it during the working call, once the scope above is written down.