Egocentric Video Data for Robotics Startups

A startup’s training pipeline typically needs first-person video of the specific tasks its product performs — manipulation, navigation, or tool use in the scenes and with the objects the product will actually encounter — rather than a broad general-purpose corpus. The streams that matter (RGB alone, or RGB plus depth, IMU, and hand pose) and the labels needed (action segments, hand pose) depend on what the training approach consumes, and are written down in a skill spec before capture starts.

Because an early-stage team is often iterating on the task list itself, the spec does not need to describe a finished, permanent program — a focused written brief covering the current tasks, scenes, and required streams and labels is a normal starting point, and the spec can be revised as the product definition firms up.

The same spec format applies whether the order is for a handful of hours or a much larger program: tasks, scenes, objects, contributor mix, streams, labels, and accept criteria. A startup does not need a different, simplified intake process to get started — it needs the same written spec a larger buyer uses, scoped to a smaller first batch.

Public egocentric and manipulation datasets were captured against someone else’s task list, scene set, and label scheme. Where a startup’s roadmap overlaps with that list, a public corpus can be a useful supplement; where it does not — a specific manipulation task, a specific object set, a specific environment — no amount of searching produces footage that was never captured for that task.

Custom collection starts from the startup’s own written spec instead, so the tasks, scenes, objects, and labels captured are the ones the roadmap actually needs, and the license terms are the ones the startup needs to ship a product rather than the research-only terms many public corpora carry.

This does not mean public datasets are irrelevant to an early-stage team — where a public corpus already covers part of the task list under a usable license, there is no reason to recapture that portion. Custom collection is scoped to fill the gap: the tasks, objects, regions, or contributor demographics a public release does not already cover, rather than replacing everything a team already has access to.

Rather than committing to full volume on a spec that has never been run against real footage, a startup can order a Spec Pilot: a small, fast first batch captured against the written spec, QA’d, and delivered with the same reject log and coverage reporting a full order gets. Billing for a pilot is per validated hour, the same unit as a full collection, but at pilot volume, so the cost of discovering a spec needs correction is small.

Once the pilot is reviewed and accepted, the same written spec — corrected for anything the pilot surfaced — scales directly into the full order, without restarting the specification process. That gives a small team a real, inspectable batch of its own data before deciding how much to commit.

A custom order delivers episodes structured and labeled exactly as the written spec defines — the streams specced (RGB, plus depth, IMU, and hand pose where ordered), the labels specced (action segments, hand pose), and the delivery format the training pipeline expects. Episodes ship alongside a QA report and reject log, the same reporting any custom collection gets regardless of order size.

Every contributor signs a documented consent release before capture, and that consent record travels with the episode through delivery; anonymization of bystander faces, plates, and screens is applied as a standard part of the pipeline, not an optional add-on for smaller orders.

Delivered data carries a perpetual, buyer-owned, non-exclusive license by default — not a hosted-access or research-only arrangement. For a startup, that distinction matters because a model trained on research-only data typically cannot ship in a commercial product without renegotiating rights, which is a risk a small team cannot always absorb once a product is already built on that model.

Because the license is settled at the time of delivery rather than negotiated later, a startup can factor the terms into its own cap table and investor conversations from day one, instead of discovering a licensing gap after a model is already trained.

Custom egocentric data delivers in RLDS, LeRobot, WebDataset, HDF5, zarr, or Rerun .rrd, or as raw streams with metadata — the same set of formats available to any custom order, not a reduced set for smaller buyers. RLDS and LeRobot in particular are built around episodic, step-based sequences that drop directly into common imitation-learning and policy-training pipelines.

Matching the delivery format to whatever a startup’s pipeline already consumes avoids building a one-off data loader just to use a custom dataset, which matters more for a small engineering team than for a larger lab with dedicated data-infrastructure headcount.

Billing is per validated hour, the same unit used for any custom collection regardless of buyer size. What moves the quote is the spec itself: how many streams are captured per episode, how deep label coverage needs to be, how broad the reach (scenes, objects, environments) needs to be, and turnaround — not a separate pricing tier for early-stage buyers.

There is no published list price because the spec determines the quote; a pilot at small volume is a way to see the real cost and turnaround against a specific spec before committing to a larger order.

Once a pilot batch is accepted, the same written spec — corrected where the pilot surfaced an issue — scales directly into a full order, carrying forward the same streams, labels, and format already proven. There is no separate re-specification step required to move from pilot volume to production volume.

As the product and task list evolve, the spec can be revised and a new pilot run against the updated version, so a startup’s data program grows alongside the product rather than being fixed at the outset.

Capture, review, and delivery continue to run in batches as volume grows, rather than as one large drop at the end of a program. That means a startup scaling up from a pilot sees validated data and passing QA arriving incrementally, which matters for a small team that needs to start training on early batches while later batches are still in review.