Glossary
What Is a Pilot Data Collection?
Last reviewed: September 2026
Short answer
A pilot data collection is a small, fast first batch captured against a written skill spec to prove the spec and the accept criteria before scaling to full volume. Firsthand calls this a Spec Pilot: it surfaces reject rate, coverage depth, and stream count on real footage, at low volume, before a larger commitment.
01
Why it matters
Committing to volume on a spec you have never seen executed is a bet on the vendor’s pitch deck, not on data. A pilot turns that bet into a measured result — the exact reject rate, sync numbers, and coverage fill you would get at scale — while the cost of being wrong is still small.
02
How Firsthand handles it
Firsthand runs a Spec Pilot as a small, fast first batch against your written skill spec, delivered with the same validated-hour billing, reject log, and coverage reporting a full custom collection gets, so you can score it on the same weighted scorecard used to evaluate any vendor before you scale volume on demand.
03
Pilot vs. full custom collection
| Property | Pilot | Full custom collection |
|---|---|---|
| Purpose | Prove the spec and accept criteria | Deliver the target validated hours |
| Volume | Small, fast first batch | Full spec’d hours |
| Billing unit | Validated hour | Validated hour |
| Reporting | Reject log, coverage against spec | Reject log, coverage against spec, per batch |
| Next step | Score on your vendor scorecard, then scale | Ongoing or one-time delivery |
04
Frequently asked questions
- What is a pilot data collection?
- A small, fast first batch of footage captured against a written skill spec, used to prove the spec and accept criteria at low volume before committing to a full custom collection.
- Is a pilot the same thing as a free sample?
- Not necessarily. A free sample pack shows existing footage; a pilot (Firsthand’s Spec Pilot) is a small, fast first batch captured against your own written spec, so it proves your spec specifically rather than a generic example.
- How is a pilot billed?
- The same way as a full collection: per validated hour, not per raw hour recorded.
- What should I check in the pilot results?
- Score the delivered footage on the same weighted scorecard you would use for any vendor — sync error, annotation depth, license, consent, reject policy, and billing unit — before committing to volume.
- Do I need a pilot before every custom collection?
- No standing program or pilot is required to start. A focused brief is a normal starting point, and many teams choose a Spec Pilot first specifically to prove a new or unfamiliar spec before scaling.
Related terms