Consistent Egocentric Data Supply for Robotics Labs

Consistency is not volume. A lab can receive a large delivery once and still have an inconsistent supply if the next delivery uses a different camera tier, a different task definition, a different file layout or a different quality bar. For a robotics team, that inconsistency shows up later as a training run that cannot mix the new data with the old, a reader that breaks on a new layout, or an evaluation that drifts because the labels changed.

This page therefore defines a consistent supply by four properties that stay fixed across batches: the written skill spec, the QA bar, the delivery format and the licence. Volume and cadence are then set against those fixed properties. Where the site states a figure for volume or cadence it is quoted here; where it does not, the page says Not stated and leaves the number to be quoted against your spec.

An off-the-shelf catalog is fast because the data already exists. That is also its limit: it covers the environments, devices and tasks that were captured when the dataset was built, and a lab’s specific task list rarely matches that exactly. A custom collection can match any spec, but a first batch still has to be captured, so it is the slower path to the very first hours.

Combining the two lets a lab start training on catalog data while the custom spec is written and piloted, then move the weight of the training mix toward custom batches as they arrive. The catalog gives immediate volume; custom collection at scale closes the gaps the catalog does not cover and keeps adding to them. The site’s comparison page sets out the same trade-off for a single decision; this page covers how the two combine over time.

The off-the-shelf library is organised into categories: virtual tour and 360° image data, household egocentric video, phone mono egocentric video, industrial and skilled-trade video, stereo and hand-tracking video, and multi-view egocentric video. Each dataset in a category lists its device, environment, region, volume and licence terms, and each has a sample folder and a metadata sheet that can be reviewed before ordering.

Two properties matter for supply planning. First, the catalog is a fixed inventory: it is what is in stock, not a flow, so it supplies a starting volume rather than a recurring stream. Second, licence terms are set per dataset and differ from the custom licence, so the catalog and custom batches should be checked against each other before they are mixed in one training run. Pricing for catalog data is a custom quote from a representative, based on hours ordered, dataset mix and licence terms.

On the stereo side, the catalog lists four stereo datasets, and only dataset 06 is stated as delivered in MCAP. The remaining stereo datasets are not stated as MCAP, so a lab that needs MCAP across the whole supply should request it for the custom batches.

Custom collection at scale runs through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The site states no provider count beyond that wording, and no hours or throughput for the network as a whole. What it does state is how the work is organised: a written skill spec, a Spec Pilot that proves it on real footage, and then batches captured to that spec on the same rigs across LATAM, Africa, Asia, the USA and Europe.

Gaps are written as spec cells. A cell has a clear start, end, outcome and object set, for example a particular task in a particular kind of workplace. Where a cell is meaningfully new it is piloted on its own small batch before it is scaled, and where it closely matches a cell already proven it is folded into the next scaled batch. The data is real human egocentric capture, never synthetic.

Because capture is spread across a large provider network rather than concentrated in one site or one crew, coverage across environments and regions is a property of the network, not a bespoke build for each order. The sibling pages on large-scale collection and on the contributor network describe the scale and the contributor workflow in more detail, and this page deliberately does not repeat them.

The site publishes one per-batch speed claim: a median of 48 hours from signed spec to first validated batch where coverage is deep, and three to four weeks where coverage is thin. That is a per-batch figure and it is never promised faster.

The site also states two program milestones, both assuming capture coverage is already in place: a pilot reaches its first delivery in two to three weeks, and a standing program reaches full rate in four to six weeks. These are milestones for the program, and they sit beside the per-batch figure rather than replacing it.

Batch size, batches per week or month, and later-batch turnaround are Not stated for a general program and are quoted against your spec. The one weekly figure on the site belongs to a single named program: the Insta360 everyday-routines program delivers 1,000 videos a week in weekly batches with metadata, consent records and a reject log. That figure belongs to that program only and should not be read as a cadence for any other collection.

Billing stays on one unit throughout: per validated hour, per batch. Cost therefore tracks the validated hours delivered in each batch rather than a lump sum for the engagement.

The spec grows additively. Each new task or failure case becomes a new cell appended to the existing spec; earlier cells are not rewritten. A model failure observed in testing is turned into a spec cell, captured in the next batch, and recorded in the data card that ships with that batch, so each turn of the loop is documented and the lab can see exactly which cells a batch covered.

QA does not reset between batches. On the instrumented rig, every session is calibrated before capture and again after it, per-camera intrinsics must land under 0.28 px RMS reprojection or the session does not start, and cross-stream sync is hardware-triggered under a 2.0 ms ceiling. Label and annotation review applies where hand pose and action segments are included. Episodes that fail are rejected, and a reject log accompanies each batch so the lab can see what was screened out in that batch specifically.

The effect is that the tenth batch is held to the same bar as the Spec Pilot that validated the spec. That is the practical definition of a consistent supply: the bar is fixed first and the volume is fitted to it.

Firsthand offers MCAP delivery. MCAP is an open-source container file format for multimodal log data, documented at mcap.dev and maintained in the foxglove/mcap repository, which publishes libraries and a command-line tool under an MIT licence. A container that holds several timestamped streams in one file suits a recurring supply, because each batch can be read by the same reader without per-stream file juggling.

For egocentric data, the streams the site already states can be mapped into a container: the stereo pair and hand pose. The site does not publish MCAP topic names, message schemas or rates, so none are stated here. The topic layout is agreed per order by naming the streams your reader expects, and then held constant across batches. The only stereo clip on the site is a rig example and is not an MCAP file.

The same episodes can also ship as RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or as raw streams with metadata. Whatever is chosen is set once in the spec and kept for every later batch, so the lab integrates a format one time.

Supply usually degrades in a small number of predictable ways. The first is calibration drift, where a later batch is captured on a rig whose intrinsics or sync no longer match the first. The site handles this by calibrating every session twice and refusing to start a session whose intrinsics miss the RMS limit. The second is format drift, where a later delivery changes layout. Fixing the format in the original spec prevents it.

The third is spec drift, where a task definition is reworded between batches so older and newer data no longer mean the same thing. Appending cells instead of rewriting them keeps older cells comparable. The fourth is thin coverage, where a region or environment has fewer capture providers available, which is why the site states the slower three-to-four-week figure for thin coverage instead of the 48-hour median.

The fifth is silent quality loss, where weaker episodes are delivered with a caveat. The site does not do this: failing episodes are rejected and listed in the reject log. A lab reading successive reject logs can see whether rejection is rising, which is an early warning that a cell needs re-piloting.

Start by taking what is already in stock. Review the stereo and hand-tracking category and any other category that matches your environments, request the sample folder and metadata sheet, and train a first model on that volume. In parallel, write the gap list: the tasks, environments and streams the catalog does not cover.

Turn the gap list into a written skill spec and run a Spec Pilot. The pilot is QA’d and delivered with a reject log and coverage report, so the spec and accept criteria are confirmed on real footage before volume is committed. From there, scope batches against the spec, name the delivery format, including MCAP if your reader needs it, and let new cells extend the spec as testing exposes new gaps. Cadence and batch size are quoted against your spec.