On-Demand Data Collection vs Dataset Marketplaces

Dataset marketplaces and on-demand collection solve different problems. In the generic sense used on this page, a dataset marketplace is a place where existing footage or episodes are listed and sold, usually under the seller’s own terms. You choose from what has already been recorded. On-demand data collection starts from the opposite end: you describe a gap in your model, the data is captured to a written spec after you commission it, and it is delivered in batches under a stated licence. See /custom-collection/on-demand-data-collection for how Firsthand runs it.

This page describes marketplaces only generically. It names no marketplace, exchange or vendor, and it quotes no third-party specs, rates or prices, because listings differ and change. Wherever the marketplace side appears, the page says what usually varies and what to ask. Anything the site does not state is marked not stated.

The comparison is distinct from two nearby pages. Custom vs off-the-shelf robotics datasets (/compare/custom-vs-off-the-shelf-robotics-datasets) compares bespoke datasets with pre-packaged ones. Public vs custom pretraining datasets (/compare/public-vs-custom-pretraining-datasets) compares open research datasets with custom pretraining collection. This page is about the buying channel: commissioning egocentric capture to your spec versus purchasing licensed footage that a third party has listed for sale.

The practical difference is who decides what exists. On a marketplace, the seller decided: the tasks, cameras, streams, labels and licence were fixed before you arrived. In an on-demand engagement, you decide, and a pipeline of recruited contributors, hardware, QA and consent paperwork already exists to carry it out. The median time from signed spec to first validated batch is 48 hours where coverage is already deep, and three to four weeks where contributors have to be recruited first.

A marketplace listing is usually written for a general buyer, so it describes content, meaning what happens in the footage, more than it describes what a robot-learning pipeline needs. The gaps below are things to check, not claims about any particular listing, and each one varies by seller.

Calibration comes first. Intrinsics and extrinsics let you undistort, rectify and project between cameras. A listing that supplies none leaves geometry as an assumption. Timing is second: if streams are aligned in software after the fact, you cannot tell how far they drift, and a stated ceiling with per-stream timestamps in the data is the thing to look for. Third are streams beyond RGB. Depth, motion and hand pose recorded at capture time are different from the same quantities inferred later from pixels, and a listing should say which it is.

Fourth is the task spec. Without accept and reject criteria and coverage targets, an hour count tells you how much footage exists but not how much of it is on task. Fifth is the consent chain and bystander handling: you want the release text, how it is stored, and how faces, plates, screens and audio were handled before distribution. Sixth is licence reach: whether the terms cover commercial model training and whether the rights pass to you as a downstream buyer.

None of this is impossible for a marketplace seller to provide. It is simply not guaranteed by the channel, so you have to ask for each item and compare the answers with the table above. In an on-demand engagement, each of them is a line in the spec before capture starts.

On a marketplace, you control selection, not content. You can filter, preview and compare, but the tasks, environments, camera placement and failure cases are whatever the seller captured. If your model fails on a specific condition, a listing that happens to include it is a stroke of luck rather than a plan.

On-demand collection puts that control in a written spec. The elements are the named skill list, coverage targets, capture tier, annotation depth, accept and reject criteria, jurisdictions, delivery format and a pilot. See /custom-collection/spec-driven-egocentric-data-collection for how a skill spec is structured. Capture hardware is also a spec choice: a phone mount, a 360 head-mount or the full instrumented rig, with the hardware shipped and briefed to contributors.

Coverage control includes the awkward cases. Firsthand ships transit, occlusion and low-light footage rather than trimming it, and publishes a reject log so you can see what was removed. For a robot that has to cope with those conditions, data that has been cleaned of them is a different dataset.

Control also extends over time. Failure modes change after every training run, so on-demand top-ups are re-briefed against the same spec and accept criteria, and the new data slots into the same schema as what you already hold. A marketplace gives you no equivalent: a later purchase is a different seller’s spec, with its own cameras, labels and terms.

These three decide whether footage is usable as robot-learning data or only as video. Firsthand’s full rig records a global-shutter 4K head camera at 60 fps with a 120° lens and two 1920×1080 wrist cameras at 60 fps. Depth is recovered from the calibrated stereo pair at 848×480 and 30 fps, in metric millimetres with a per-pixel confidence. A 6-DoF IMU records at 200 Hz on the head and on each wrist. Per-tier stream lists for the phone mount and 360 head-mount are not stated, so the stream specs here describe the full rig.

Calibration is checkerboard-based: intrinsics and extrinsics are solved at the start and end of every session, reprojection error is under 0.28 pixels, and drift is written to the episode metadata. Delivering both the opening and closing solves is what lets you check drift yourself. A marketplace listing may supply calibration, a single calibration, or none; unless it says so, treat the geometry as not stated.

Time sync works the same way. Every stream is hardware-triggered onto one PTP clock, with a median of 1.12 ms and a 2.0 ms ceiling, and each stream carries int64 nanosecond timestamps so alignment can be checked from the data. Software-aligned timestamps from separate device clocks can drift over a session, and a listing that describes sync only in adjectives gives you nothing to verify.

Hand pose is 3D with 21 joints per hand, in metres, in the head-camera frame, with a per-joint visibility flag. The head IMU is head-motion context, not a camera pose. If a listing offers hand pose, ask whether it was recorded at capture time or estimated afterwards from 2D pixels, because the two behave differently around occlusion.

Firsthand delivers frame-accurate action segments with a verb, a noun and nanosecond timestamps, drawn from 97 verbs and 512 nouns, plus contact events marking the moment a hand meets an object. Per-episode success or failure labels, reward values and preference labels are not stated on the site, so this page does not claim them.

QA is checked per episode against the signed accept and reject criteria rather than sampled from a corpus. Episodes that fail are recaptured, the reject log is visible to the buyer, and rejected items are recaptured at Firsthand’s cost. You pay only for what clears the criteria, and the criteria are in the spec you signed.

On a marketplace, labels and QA depend entirely on the seller. The useful questions are concrete. Were the labels produced at capture time or afterwards, and by whom? What vocabulary were they drawn from, and how precise are the timestamps? Was every item reviewed or only a sample, and can you see what was rejected? A listing that cannot answer these is selling footage, not a labelled training set, and the price comparison should reflect that.

In an on-demand engagement, consent starts before capture. Each participant signs a written release, and consent artefacts and a data card accompany every batch. The data card lists jurisdictions of capture, which is the provenance record that travels with the data. Before anything leaves QA, faces, plates and screens are blurred and audio is scrubbed, and clips that contain bystanders who cannot be de-identified are discarded.

The licence is perpetual, irrevocable, buyer-owned and commercial. That is a statement about ownership and use. Exclusivity terms, meaning whether anyone else can receive the same footage, are not stated on this page, so ask for them in the engagement terms if they matter to you.

Marketplace terms are set by the listing and differ between sellers. The points to establish are the term of the licence, the field of use, whether commercial model training is covered, whether rights pass to a downstream buyer, whether redistribution is allowed, and whether exclusivity is on offer. Provenance questions are equally concrete: who recorded the footage, under what release, in which jurisdiction, and how bystanders and screens were handled.

For frontier AI training, where a data audit can ask for exactly this paperwork, the difference is practical. A consent artefact and data card per batch can be produced on request. Whether a marketplace seller can do the same depends on how the footage was originally gathered, which is why the question belongs at the start of the purchase rather than the end.

A marketplace purchase is a reasonable choice more often than a vendor page would like to admit. If you are exploring whether egocentric video helps at all, or smoke-testing a data pipeline, existing footage that is available now can be better than waiting for a spec. The same is true for broad pretraining where the exact task matters less than variety, and for RGB-only work where depth, motion and hand pose are not needed.

It also works when the listing documents what you need. If a seller states its camera hardware and mounting, supplies calibration, states its sync method and ceiling, gives a clear consent and anonymization process, and offers a licence that covers commercial training and reaches you as the buyer, then the main risk is task fit, which you can judge from a preview. A well-documented listing that matches your task is a good purchase.

The honest limit is fit and verification. If the footage does not cover your failure condition, no amount of documentation fixes that. If the documentation is thin, you are accepting unknowns in geometry, timing, consent or licence that you will carry into every model trained on the data. Weigh those unknowns against the time saved.

On-demand collection pays off when the gap is specific. A manipulation skill that fails in one environment, a condition the model has never seen, a population or region that is missing: these are briefs, not search terms. You pay for the hours that close the gap and stop when the evaluation moves, with no standing program required.

It pays off when the stream list matters. If your model uses depth, wrist or head motion, or 3D hand pose with visibility flags, capturing them at record time on a calibrated, synchronized rig is different from reconstructing them afterwards. It pays off when you need a documented consent chain and a buyer-owned commercial licence, for example when the data goes into a model you will ship or into a frontier training run that will be audited.

It also pays off when consistency over time matters. Top-ups against the same spec keep cameras, labels, field names and licence terms identical across purchases, so training sets grow without conversion work. Many teams combine the channels: a marketplace dataset for breadth, and an on-demand collection for the specific gap the broad data does not cover.

The cost of on-demand is that you have to write a brief. Pricing is quoted per engagement against the spec, and no prices are stated on this page. A free sample pack is available to inspect the schema before you commit.

Start from the gap, not the finished spec. Describe the task, environment or population the model is missing. That becomes a written skill spec with the elements in the stage table above. State the reject bar precisely, because that is where specs succeed or fail: whether a clip is unusable when the hands leave frame, when depth fails, when an undocumented bystander appears or when the wrong task is performed. Every episode is then checked against it.

State the format your training pipeline expects: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, each carrying the same calibration, timestamps and annotations. Then run the Spec Pilot, a small, fast first batch that proves the spec and QA bar on real footage. Use it to check stream quality, annotation fidelity and format compatibility with your own pipeline before committing to volume. If something is off, the spec changes, not the data.

After the pilot passes, the same pipeline runs batches against the spec. For speed and what governs it, see /custom-collection/fast-data-collection. For cost drivers, see /custom-collection/pricing. What this page does not state: prices, per-tier stream lists, contributor counts or any throughput figure.