On-Demand vs Ongoing Data Collection Programs

On-demand data collection is custom data gathered when you need it, against a spec you set, from contributors who are already recruited, briefed and paid through a standing pipeline. You describe a gap, you buy the hours or items that close it, and you stop when your evaluation moves. No standing program, retainer or minimum is required to start.

An ongoing data collection program takes that same pipeline and keeps it running after the first Spec Pilot, as a series of batches. Each batch runs against the same written skill spec, which grows as new tasks or failure cases are added, passes the same QA bar, and ships under the same buyer-owned licence. The unit of work does not change: it is still a validated hour of first-person video captured under a written spec. What changes is the cadence and the posture. On demand, you brief each time a gap appears. In an ongoing program, the spec and pipeline stay in motion between batches.

That also answers the one-off versus continuous dataset collection question. A one-off order answers a fixed brief and closes once delivered. A continuous program does not close after one delivery. On-demand sits closer to the one-off end, but it is not strictly single-shot: top-ups can be re-briefed against the same spec whenever the model’s failure modes shift.

Both run on one pipeline. A written skill spec defines the task, the coverage targets and the accept and reject criteria. Capture runs against that spec. QA checks every episode against the signed criteria, anonymization and consent are applied, and the data is delivered under a licence the buyer owns. The site describes an ongoing program as the same written-spec, Spec Pilot, QA and licensed-delivery pipeline run repeatedly, so none of these stages is specific to one model.

The practical consequence is that choosing between them is not choosing between a rigorous process and a loose one. On-demand does not mean lower standards: every item clears the signed accept criteria, anonymization and consent checks before delivery. The choice is about how often the pipeline runs and how much of the spec you expect to carry from one run to the next.

Delivery is the same too. Either model can deliver in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, and each batch carries a data card. The formats are described in the official documentation linked under Sources.

A Spec Pilot is a small, fast first batch captured against the written skill spec, used to prove the spec and its accept criteria before volume is committed. It is not a free sample of footage that already exists: it is captured against the exact spec a full collection would use, which is why its reject rate, coverage fill and sync numbers are a fair preview of what larger batches will look like.

An ongoing program starts exactly the way a one-off order does, with a written spec proven on a pilot. What makes it a program is the decision, after that first pilot is accepted, to keep the same spec and the pipeline that validated it running as further batches are scoped. The pilot is delivered with a reject log and a coverage report, the same reporting a full batch gets.

A pilot is optional. A buyer who has run a similar collection before, or whose spec covers a familiar task, often moves straight to full volume, while a buyer testing an unfamiliar condition mix gets more value from a small batch first. Inside an ongoing program the same rule applies to each new spec cell: pilot it on its own small batch if it is meaningfully new, or fold it into the next scaled batch if it closely matches a cell the spec has already proven. For the full pilot-to-production path, see the scaling guide linked below.

On demand, a change is a re-brief. Failure modes shift after every training run, so you describe a new condition, region or edge case and run it against the same spec and the same accept criteria. The new data is consistent with what you already hold and slots into the same schema.

In an ongoing program, change is built into the cadence. Each new requirement, whether a new task, a failure case a deployed model surfaced, or a scene or object variant that was thin in earlier coverage, is written up as a new cell added to the existing spec, with a clear start, end, outcome and object set. Prior cells are not rewritten: the spec grows additively. Later batches do not need to re-run the full specification process from zero, because only the new or changed portion has to be written up and piloted.

The data flywheel is the loop the ongoing model is designed to support: a model is tested or deployed, its failures are observed, those failures are written up as new spec cells, targeted data is collected, and the retrained model is tested again. The data card that ships with each batch records which cells were captured and what was rejected, so every turn of the loop is documented.

Brief to spec takes the same week. From a signed spec, the first validated batch arrives in a median of 48 hours where coverage is already deep, and in three to four weeks where contributors have to be recruited first. The same two figures appear on the on-demand page, the ongoing program page and the data flywheel definition, and this page does not promise anything faster.

What is not stated is the turnaround of the second and later batches of an ongoing program, the batch cadence, the volume per batch, and any minimum term. The site says that batches recur and that the pipeline is already proven after the first pilot, but it gives no schedule and no figure. If any of these matters to your plan, ask for it as part of the spec conversation rather than assuming it from this page.

Quality is held constant by design. Every batch in an ongoing program passes through the same calibration and synchronization checks on every active stream, plus label and annotation review for hand pose and action segments where those are included. Episodes that fail are rejected rather than delivered with a quality caveat, in every batch and not only the first, and a reject log accompanies each batch so you can see what was screened out in that batch specifically.

Consent works per session. Every contributor across every batch signs the same consent release that governs a one-off order, and each session carries its own consent record through delivery, so consent is not renegotiated at the start of each batch. Anonymization is applied with the same scope used elsewhere on the site, per session, as part of the standard pipeline.

The licence does not need to be re-established either. Delivery across a program ships under one perpetual, buyer-owned commercial licence, so each new batch becomes a permanent addition to a training pipeline the buyer already owns outright. Exclusivity terms are not stated on this page, so confirm them in the engagement terms if you need them.

On-demand work is quoted per validated hour or per accepted item, against the spec, before any capture. You pay only for what clears the accept criteria, and rejected items are recaptured at Firsthand’s cost. An ongoing program keeps the same unit: per validated hour, per batch, rather than a single lump-sum price for the whole engagement. That ties each batch’s cost to the validated hours it delivers, the same way a single pilot or one-off order is billed.

No prices are stated on this page, and no discounts or tiers for recurring work are stated anywhere it relies on. Cost drivers are described on the pricing page, linked in the related pages below.

On-demand fits a specific, bounded gap: one environment where a skill fails, a condition the model has never seen, a region or population that is missing. It fits a team that wants to pay for the gap rather than a program, and it fits a fixed, known need that will not change much once the data is delivered. When the evaluation moves, you stop.

An ongoing program fits a model that is being trained, evaluated and deployed continuously, where testing keeps surfacing new failure cases and coverage gaps. If you already expect to need more data next month, in the same format, against a spec that will keep growing, the program removes the need to treat each requirement as a separate order. It also gives you one QA bar, one consent process and one licence across all of it.

The two are not exclusive. A common path is to start on demand with a single brief and a Spec Pilot, then decide after the pilot is accepted whether to keep the same spec running as an ongoing program. Where you are unsure, the pilot is the cheapest way to find out, because it shows real reject and fill numbers before the full budget is committed.

Both start the same way: a written brief covering the tasks, scenes, objects and streams in scope, turned into a skill spec and proven on a Spec Pilot before volume is committed. State the reject bar precisely, and name the format your training pipeline expects.

If you expect the work to continue, say so up front. Flagging that the engagement is expected to run as a series of batches means the spec, the consent process and the delivery format are set up to extend cleanly as new cells are added later. If you are not sure, start on demand: nothing in the on-demand route prevents continuing afterwards.