In-Home Egocentric Data Collection

In-home egocentric data collection means recording first-person video inside real homes, from a viewpoint close to what a household robot’s own camera would see. A paid contributor wears a capture rig and performs a household task from a written skill spec. The footage is checked, anonymized, labeled and delivered to your training stack.

It is human capture. A person wearing the rig does the task, so the data is first-person human video of home tasks rather than robot trajectories logged in a home. A person can move through many homes without the safety and access problems of taking an early-stage robot through someone’s kitchen or bathroom.

A household robot deployed at scale will work in homes it never saw during training: different layouts, furniture, clutter and people. A model trained on a staged set or a single house learns that set or that house.

NVIDIA’s GR00T N1 paper describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of home tasks sits in that base layer. Human capture is the practical way to cover many homes because it does not need a robot on hand in every home visited.

Diversity matters as much as hours. Generalist’s GEN-0 write-up states that data quality and diversity matters more than sheer volume. A corpus that repeats one living room many times covers less than the same hours across many homes, which is why the spec names room types rather than leaving coverage to chance.

The household tasks this site already documents are laundry, tidying, cleaning and surface reset, across whole homes. They are long-horizon, low-precision and navigation-heavy, which is why they differ from tabletop manipulation. The first table lists the six stated task categories and what each one exercises.

The same categories span several room types: living space, bedroom, bathroom, utility room, and stairs and hallway. Kitchens are covered separately on this site, where the kitchen dataset describes two-handed food prep in real homes. The second table lists each setting and how the spec treats it.

The scope stays household chores; anything not stated here is set per program in the skill spec.

We design the first draft of the spec with you on a call. For in-home work it names the task list, the room types the corpus must span, the participant and lighting mix, the capture tier, annotation depth and the accept criteria.

Room types should be named explicitly, because a laundry cycle or a cleaning routine crosses rooms rather than staying in one cell. Coverage targets can be set per axis (contributor, scene, task, object, condition and failure case) and tracked per batch, as described on the diversity page.

How many homes, rooms or contributors a program covers is set per program in the skill spec, and the quote states reach, timeline and price before any capture begins.

The instrumented tier is a head-mounted stereo rig. It records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider capture stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms.

Every session is calibrated twice, before capture starts and after it ends. The solve uses a checkerboard target across 40 poses, per-camera intrinsics to a reprojection error under 0.28 pixels, and head, wrist and depth extrinsics against the shared target. The drift between the two solves is written into the episode metadata.

Labels follow the delivered episode schema: hand_joints with hand_visible flags, body_joints, contact_events, instance_masks, objects, action_segments and gaze_target. Which of these a program attaches is set in the skill spec. The third table maps each part of the setup to where it is recorded. The rig is worn by the contributor; no robot or teleoperation capture is part of this page.

Household capture happens inside lived-in spaces, which raises the bar above a lab or workshop. Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. The consent chain has six checkpoints, from before capture to after delivery, and a break at any one discards the footage rather than shipping it at a lower grade.

At ingest, automated detection finds faces, plates and screens and blurs them, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame. If a person cannot be de-identified, the footage is discarded.

Bathrooms get a stricter rule: capture excludes any episode with a person in frame other than the consenting operator. Site permission is kept on file where the site needs it, and the exact permission detail for your homes is set per program in the skill spec. The provenance page publishes the release and the anonymization page describes the full pass. None of this is legal advice, and the executed agreement controls.

Validation runs against signed accept criteria written into the spec before capture. A reject log ships with every batch. An episode is accepted only if RMS reprojection stays under 0.28 px and the drift is within tolerance; otherwise it goes to the reject log, visible to you.

At ingest, episodes are machine-checked against the reject criteria in the fourth table, and a human reviewer then checks each episode against the task spec. Rejected items are recaptured at our cost rather than shipped with a caveat.

Most orders start with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before volume scales.

Public datasets are captured for someone else’s task distribution. The fifth table compares Egocentric-10K, whose official card states the footage is collected exclusively in real factories, attribute by attribute; where the card does not state something, the table says so.

The structural difference is control over the setting: a custom in-home collection is scoped to the room types and tasks your model is missing. A public release can still suit exploration or a baseline. Whatever you use, check its license and consent terms.

Delivery is in the format your training stack reads: RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. Converter source is readable, so you can retarget the schema to your own loader. The HDF5 layout, including the stream names, is on the custom HDF5 page and the episode schema page.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted. Exclusivity is available per skill line or per environment as a separate term, and the executed agreement controls.

Describe the rooms, home layouts or tasks your model is missing. That becomes a written spec, quoted for reach, timeline and price before any capture begins. Many programs start with a Spec Pilot.

Where coverage for your tasks and regions is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, three to four weeks while recruiting expands. Anything this page does not state, such as home or contributor counts, is set per program in the skill spec.