Glossary
What Is On-Demand Data Collection?
Last reviewed: September 2026
Short answer
On-demand data collection means a buyer specifies exactly the tasks, scenes, objects, and conditions a model needs, and a provider captures that data when the gap appears — rather than the buyer choosing from a fixed catalog of pre-existing data or standing up its own recruiting, capture, and QA operation. The provider’s contributors, hardware, and QA process already exist; a new brief points that pipeline at a new gap instead of starting from zero.
01
Why it matters
A fixed catalog dataset was built to someone else’s spec and may not cover the exact scene, task, or population a model is failing on. Building an in-house capture operation to close that gap means recruiting and paying contributors, sourcing or building capture hardware, and staffing annotation and QA — all before a single usable clip exists. On-demand data collection sits between the two: the buyer keeps control of the spec and accept criteria, the provider supplies the standing pipeline, and the buyer pays for the specific hours or items that close the gap rather than funding a catalog subscription or a long-running internal program.
02
How Firsthand handles it
Firsthand runs on-demand data collection as a warm pipeline: contributors are already recruited, briefed, and paid through the same consent and QA flow used on every engagement, so a new brief does not start from zero. A signed spec typically reaches a validated first batch in a median of 48 hours where coverage is already deep, or 3–4 weeks where contributors need to be recruited first in a thinner environment. There is no standing program requirement — a focused brief on one condition, one region, or one missing scene is a normal starting point, often opening with a small pilot batch before scaling. Every item still clears the same signed accept criteria, anonymization, and consent checks as a full program, and delivery arrives under a perpetual, buyer-owned license in the training stack’s existing format.
03
On-demand vs. off-the-shelf vs. in-house
| Who owns it | On-demand collection | Off-the-shelf dataset | In-house pipeline |
|---|---|---|---|
| Spec | Buyer writes the brief; provider turns it into a spec with accept criteria | Fixed by the dataset publisher | Buyer writes and maintains it |
| Coverage | Scoped to the exact gap the buyer names | Whatever the publisher already captured | Whatever the buyer chooses to build toward |
| Time to first data | A median of 48 hours where coverage is already deep | Immediate, but fixed and non-negotiable | Weeks to months to stand up recruiting and QA |
| Licensing | Perpetual, buyer-owned license on delivery | Publisher’s license terms, often non-exclusive | Buyer already owns everything captured |
| Standing commitment | No minimum program; top up as needed | One-time purchase of a fixed catalog | Ongoing headcount and vendor management |
05
What happens when you brief an on-demand gap
| Stage | What happens | What the buyer receives |
|---|---|---|
| Brief | The buyer describes the gap — a task, scene, or population the model fails on | A spec with accept and reject criteria to sign off on |
| Pilot | A small batch is captured first against the same spec | Early validated data to confirm coverage before scaling |
| Capture | Already-recruited contributors are briefed and pointed at the new spec | Footage or sensor streams captured against the brief |
| QA | Every item is scored against the accept criteria | A visible reject log, not a downgraded delivery |
| Delivery | Data is packaged into the requested format with a license and data card | A finished, buyer-owned dataset |
06
Frequently asked questions
- What is on-demand data collection?
- Custom training data captured against a buyer’s spec when a gap is identified, using a provider’s already-recruited contributors and existing capture and QA pipeline, instead of choosing from a fixed catalog or building an internal collection operation.
- How is on-demand collection different from buying an off-the-shelf dataset?
- An off-the-shelf dataset already exists and was built to someone else’s spec, so it may not match a model’s specific failure mode or population. On-demand collection captures new data against the buyer’s own spec and accept criteria.
- How is on-demand collection different from running collection in-house?
- In-house means the buyer recruits contributors, sources or builds capture hardware, and staffs QA itself before any data exists. On-demand collection uses a provider’s already-warm pipeline, so a new brief does not start from zero.
- How does on-demand data collection work for robotics and physical AI?
- The same model applies to egocentric video, hand-pose, depth, and action-labeled robotics data: a provider turns a described gap into a spec, points already-briefed contributors at it, runs QA against accept criteria, and delivers a licensed dataset in a robot-learning format.
- How fast can on-demand data collection deliver?
- A median of 48 hours from signed spec to first validated batch where coverage is already deep, or 3–4 weeks where contributors need to be recruited first for a thinner environment.
- When should an AI lab use on-demand collection?
- When a specific, narrow gap appears — one missing scene, condition, or region — rather than a need for a large standing program. It also suits topping up an existing dataset as a model’s failure modes shift after each training run.
- Is there a minimum order for on-demand data collection?
- No. A focused brief on one condition or one missing region is a normal starting point, and engagements often open with a small pilot batch before scaling to full volume.
- Can on-demand data be topped up later?
- Yes. A top-up runs against the original spec and accept criteria, so new data stays consistent with what a buyer already holds and slots into the same schema.