Egocentric Datasets Delivered With a Data Card

A data card is a structured record shipped with a dataset that describes how it was collected, the consent basis, the jurisdictions of capture, the license and known limitations. That is how this site’s glossary defines the term. The point is to make a dataset auditable rather than a folder of files you have to take on trust.

For egocentric data the record matters more than usual. First-person video captures bystanders, homes, workplaces and screens, so who consented, where it was filmed and what was removed are part of the product, not an appendix. A data card keeps those answers next to the data, so your legal and ML reviewers read the same record.

This page covers custom egocentric datasets collected on demand against a written skill spec, with a data card in every batch. It is about documentation. It makes no claim that any model trained on this data performed in any particular way.

The site states a specific set of items. The data card records the jurisdictions of capture and the consent basis. It lists consent artefact IDs per episode. It records the de-identification pass, meaning the detector version, the pass timestamp and the reviewer who confirmed it. And it records which coverage cells from the spec’s condition matrix were captured and what was rejected.

The glossary also lists the reject log among the card’s contents, while other pages list it as its own item in the delivery. Either way, a reject log ships with every batch.

What the site does not state is the card’s file format, its section order or a field-by-field template, and this page does not invent one. The first table below lists only items the site already places in the data card. The second lists everything else that arrives with the batch.

The glossary says provenance is now a procurement and compliance requirement, and that a data card is what lets a buyer’s legal and ML teams sign off on training data with confidence. For a team buying custom data, the card is the document procurement and counsel can review instead of relying on a sales summary.

The Datasheets for Datasets paper makes the wider case. It notes that a model is unlikely to perform well in the wild if its deployment context does not match its training or evaluation datasets, and it proposes that every dataset be accompanied by a datasheet documenting its motivation, composition, collection process and recommended uses. The site cites that paper as the basis for the data card.

One limit applies. Firsthand captures first-person human video, not robot or teleoperation data, and capture does not take place at a customer’s own facility. A data card tells you what the data is. It does not change what it is, so bridging the embodiment gap to a specific robot is your pipeline’s job.

You describe the task your pipeline needs covered and that becomes a written skill spec: the tasks, environments, objects and streams, how many attempts, what counts as a completed attempt, and the accept criteria. The accept criteria are signed into the spec before capture begins. The site’s guidance is to name the license and the consent and provenance documentation you will receive in the spec and order form before capture, not after delivery.

Each batch then arrives with its data card, the consent artefact IDs listed per episode, the signed accept criteria it was measured against, and the reject log. The batch is judged against the standard you signed, and the card records what was captured and rejected against the spec’s condition matrix.

In an ongoing program the card documents every turn of the loop. An observed model failure becomes a new or updated spec cell, the next batch captures against it, and that batch’s data card records which cells were captured and what was rejected. Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale.

The consent chain has six checkpoints, from before capture to after delivery. The ones this site names are these: a written release is signed and identity verified before capture; site permission is on file at capture; faces, plates and screens are detected and speech is scrubbed of identifiers at ingest; a human reviewer confirms no un-consented person is identifiable at review; and consent artefact IDs are listed per episode in the batch data card at delivery.

On licensing, the data ships under a perpetual, irrevocable, worldwide, buyer-owned license that is non-exclusive by default. The data card records the jurisdictions of capture and the consent basis. The license terms you agreed, including any exclusivity, are the ones written into the order form, which the terms page says governs. So the card and the order form split the work: the card holds provenance, and the order form holds the license terms.

Nothing here is legal advice and this page makes no compliance claim. The data card lists jurisdictions of capture, and what applies to your use is a question for your counsel. The consent chain is also the ceiling a license cannot exceed. Delivered footage cannot be recalled after delivery, and the release says so, while withdrawal requests are honoured for undelivered footage.

The site does not list streams, labels or formats as data card fields, so this page does not either. They are described in three other places: the skill spec says what is captured, the episode schema lists the per-episode fields, and the delivery format pages describe each format. The third table below separates the batch-level record from the per-episode fields.

On the episode schema, meta.json carries episode_id, skill_id, duration_ns and reject_flags. The annotations include action_segments with start_ns, end_ns, verb and noun, hand_joints and hand_visible for 21 joints per hand with a per-joint visibility flag, contact_events, objects, and instance_masks at 848×480 where the spec calls for them. Streams include depth_mm, imu_head and imu_wrist_l/r, and the calibration record carries rms_reproj_px and calib_drift_mm.

You receive the accepted episodes in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or as raw streams with metadata, and readable converter source ships alongside so the schema can be retargeted to your loader. A WebDataset delivery is .tar shards of 512 samples each, plus the reject log and the batch data card, and each sample carries a __key__, head.jpg, wrist_l.jpg and wrist_r.jpg for the head and wrist views.

An episode is accepted only if RMS reprojection stays under 0.28 px and the drift is within tolerance. Episodes are machine-checked against the reject criteria, then a human reviewer checks each one against the task spec. Rejected episodes go to the reject log, which ships with every batch, and are recaptured at our cost. Only footage that clears the criteria is billed as a validated hour, and each episode’s meta.json carries its own reject_flags.

The data card ties that validation back to the spec’s condition matrix. A buyer can see which conditions are covered, which were thin, and which cells were rejected and why, because the reject log carries reasons rather than only pass counts. That is a record you can audit, not a summary you have to trust.

If the same failure keeps recurring, the more useful fix is often to revisit the spec rather than keep recapturing, for example when the geometry that must stay in frame is impossible to hold in a particular environment. The validated egocentric video data page covers the acceptance test in full.

From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. The data card ships with that first batch as it does with every batch.

The site states no separate timeline for the data card, so none is claimed here. After the first batch, on-demand top-ups run against the same accept criteria, so you can add attempts or environments without rewriting the spec. The spec is quoted for reach, timeline and price before any capture begins.

The three are related rather than rival. The Datasheets for Datasets paper proposes a datasheet for every dataset and groups its questions by lifecycle stage. The Hugging Face Hub uses a README.md in the dataset repository, called a dataset card, which it renders on the dataset page. A Firsthand data card is a per-batch record shipped with a custom delivery.

The paper says its questions are not intended to be prescriptive and that datasheets will necessarily vary by domain and workflow. The site cites the paper as the basis for the data card but does not state that the card answers every datasheet question. If you want datasheet-style answers the site does not state, such as recommended data splits or a maintenance contact, ask for them in the spec.

It is not a certification and it is not legal advice. It does not replace consent, because the consent chain is the ceiling a license cannot exceed. It does not state model results, and this page claims no customer outcome. And it does not turn human demonstrations into robot data.

What it does is record how this batch was collected, consented, de-identified and checked, so the claims about it can be tested against a specific file.