Capture methodology

The method is the product. So here it is, in full.

Anyone can point a head camera at a task. What makes footage trainable is that every stream shares a clock, every session is calibrated at both ends, every operator was trained and paid properly, and every episode that fails is rejected rather than shipped.

Sync ceiling: 2.0 ms, hardware-triggered
Median error: 1.12 ms measured
Calibration: Opening and closing solve, every session
Active operators: 287 vetted

How we capture

One clock domain, or it isn't a dataset.

Ten streams recorded on separate sensors are ten separate videos until something forces them onto the same timebase. We hardware-trigger every sensor from a single PTP grandmaster, and we publish the resulting error distribution.

A head-mounted capture rig disassembled and laid out: head camera, two wrist cameras, depth sensor, IMU pucks, cables and battery pack. — FIG 1 — Rig v3, 640 g total worn weight. Every part is off-the-shelf and named below, so you can reproduce our capture conditions.

A person wearing the head-mounted rig while chopping vegetables in a real, cluttered home kitchen.

Bill of materials
Head camera	Sony IMX585 global shutter · 120° M12	Carbon headband, 42 g
Wrist cameras	IMX290 ×2 · 100° M12	Velcro cuff, rigid plate
Depth	Intel RealSense D435if	Forehead, 18 mm below head cam
IMU	Bosch BMI088 ×3 · 200 Hz	Head, both wrists
Clock	PTP grandmaster + hardware trigger	Belt pack, 190 g
Storage	2 TB NVMe · 6 h continuous	Belt pack

Rig architecture

Four device groups, one trigger, one episode file. The clock node is the only part of this diagram that is hard.

Rig architecturerig-04 · v3 payload

Measured sync error

Absolute timestamp offset between the head camera and every other stream, sampled across randomly selected frames from shipped episodes.

Cross-stream sync errorn = 12,480 frames

12,480 measured frames. p99 = 1.94 ms. Frames above the ceiling are rejected and the segment is recaptured.

Method, tooling and the raw CSV behind this plot are in the sync measurement guide. Nobody else in this category publishes this number, which is exactly why we do.

A checkerboard calibration target on a tripod facing a camera rig in a plain room. — FIG 2 — Intrinsics and extrinsics are re-solved at the start and end of every session. Drift between the two solves is recorded in the episode metadata, so it is detectable rather than assumed.

Sync measurement

Reproduce our sync number from the data you were sent.

We publish the method, not just the figure. Every delivered episode carries per-stream int64 nanosecond timestamps in the PTP domain, so you can recompute the error distribution yourself and check our claim.

sync_check.py

# reproduce our sync figure from any delivered episode
import numpy as np, h5py

with h5py.File("EP_0117_KITCHEN_A.h5") as f:
    head = f["streams/head_rgb/timestamp_ns"][:]      # int64, PTP domain
    others = {k: f[f"streams/{k}/timestamp_ns"][:]
              for k in ("wrist_l", "wrist_r", "depth", "imu_head")}

for name, ts in others.items():
    # nearest-neighbour match each head frame to this stream
    idx = np.searchsorted(ts, head).clip(1, len(ts) - 1)
    err = np.minimum(abs(ts[idx] - head), abs(ts[idx - 1] - head)) / 1e6
    print(f"{name:10s} median {np.median(err):.3f} ms   p99 {np.percentile(err, 99):.3f} ms")

Frames measured: 12,480
Median offset: 1.12 ms
p99 offset: 1.94 ms
Spec ceiling: 2.0 ms
Clock source: PTP grandmaster in the belt pack, hardware trigger line to every sensor
Sampling: Randomly selected frames from shipped episodes, not a best-case reel

If your own recomputation disagrees with these numbers on a delivered batch, that is a defect and we will recapture it.

Calibration procedure

Calibrated twice per session, so drift is measured.

A single opening calibration lets you assume nothing moved. Solving again at the end of the session turns that assumption into a number you can filter on.

1 · Target solve: Checkerboard, 9×6, 30 mm squares, 40 poses spanning the full frustum of every camera.
2 · Intrinsics: Per-camera focal, principal point and distortion solved independently; RMS reprojection error must land under 0.28 px or the session does not start.
3 · Extrinsics: Head↔wrist↔depth transforms solved against the shared target, expressed in the head-camera frame.
4 · IMU alignment: Gyro bias and accelerometer scale estimated from a 60 s static hold plus a figure-eight excitation.
5 · Closing solve: The entire procedure repeats at the end of the session. Drift between opening and closing solve is written into episode metadata.

Rejection thresholds are applied automatically at ingest — no human decides whether a drifted session is “probably fine”.

Validation and QA

You get the reject log with the batch.

Every episode is machine-checked, then reviewed by a human against the skill spec it was captured for. Rejected episodes are recaptured at our cost, and the reasons ship with the delivery.

Sync breach: Any stream exceeding the 2.0 ms ceiling for more than 3 consecutive frames
Calibration drift: Closing solve differs from opening solve beyond tolerance
Occlusion: Manipulated object out of frame for more than 15 percent of the episode
Depth invalid: More than 35 percent invalid depth pixels in the manipulation region
Consent gap: Any bystander present who cannot be de-identified
Spec miss: Episode does not match the named skill it was captured against

Reject reasons are enumerated, counted per batch, and included in the coverage report.

A multi-monitor QA workstation showing synchronized video tracks on a timeline next to a spreadsheet of per-episode numbers. — FIG 3 — Review station. All streams on one timeline, sync error plotted underneath, spec checklist on the second display.

Operators

Vetted, trained, and paid by the hour.

A per-task bounty produces fast, rushed, unusable footage. We pay for time — including setup, calibration and breaks — because the incentive has to point at quality rather than throughput.

Applied: 4,180
Passed screening call: 1,340
Completed rig training: 710
Passed a calibrated trial session: 412
Currently active: 287

Cumulative since programme start. Roughly 7 percent of applicants reach active status; the calibrated trial session is where most attrition happens.

We recruit for the trade first and teach the rig second. Someone who already knows how to break down a carton or torque a wheel to spec produces footage a policy can learn from; someone learning both the task and the rig produces hesitation. Collector standards and pay.

An operator checking a recording on a laptop while a second person wearing a capture rig works at a vise in a garage workshop. — FIG 4 — Every session is supervised. The second operator is verifying sync and framing between takes.

Capture environments

Real rooms, recorded as found.

We do not build sets. A staged kitchen teaches a policy that counters are empty and objects are conveniently separated, which is the opposite of the distribution you need at deployment.

Home kitchen
Garage workshop
Warehouse aisle
Retail shelf aisle
Clinical training room
Construction interior

Every episode carries a site identifier, so you can hold out an entire location rather than accidentally training and evaluating in the same kitchen.

Pipeline

Five steps. Spec to delivery.

Step 03 is the one most vendors leave out, and it is the reason you are not paying for footage you will throw away.

01SPECNamed skill list, coverage targets, accept/reject criteria
02CAPTURECalibrated rigs, vetted and trained operators, real environments
03VALIDATESync check, QA pass, reject and recapture — you see the reject log
04ENRICHDepth, body pose, hand pose, action segments, instance masks
05DELIVERRLDS, LeRobot, WebDataset, HDF5 — plus the data card

Want the sync CSV and a calibration bundle?

Both ship inside the free sample pack, alongside the episodes. No form before the files.

Get sample data →Read the schema