How to Evaluate an Egocentric Data Sample

A sample is only useful if you review it in an order that exposes problems early. Start with the cheapest checks, the ones that need no tooling beyond what you already run, and move toward the ones that need judgment. The order used on this page is files and formats, streams and fields, calibration and sync, hand pose and labels, anonymization and consent, and finally the license and what the sample cannot tell you. Each step has a question you can answer from the files you hold, and each answer is either something this site already states or something to request in writing.

Before you open anything, write down what you asked for and what you received. The site describes the free sample pack as 40 episodes across 4 environments, with 8 synchronized streams per episode at native rate, about 2.4 GB, drawn from the same production batches you would license. It also describes a per-recording view, the sample data outputs, with up to eight files for each recording. A custom sample shaped to your own task is different again: what it contains, how large it is and when it arrives are agreed per request. Know which of these you are holding, because the checks below differ slightly between them.

This page covers the hands-on review of files you already have. It is not the pilot evaluation, which judges a Spec Pilot against a written skill spec and ends in a go, adjust or stop decision, and it is not a vendor scorecard. A sample shows existing footage captured for a different purpose, so it tests the schema, loader fit and label quality. It cannot prove your spec. That is the pilot’s job, and the last sections explain how to hand the one to the other.

The delivery formats stated on this site are RLDS, LeRobot, WebDataset, HDF5, zarr and Rerun .rrd. The site says the sample pack ships in every format it lists, and that each format is a projection of one episode schema with four groups: meta, streams, annotations and calibration. The first test is therefore mechanical. Point the loader your training stack already uses at the format it reads and see whether an episode loads without a bespoke parser or a reformatting pass. A sample in a format or layout beyond those listed is agreed per request, not promised.

The site states that converters ship as readable source rather than a binary, so if your loader differs you can retarget the schema yourself. If a format fails to open, keep the exact error and note which format page you were following. That turns a vague complaint into a specific question for the provider. The official documentation for each format is linked in the sources below, and it is the right place to check how a format is meant to be read before concluding that a file is at fault.

Two cautions apply to anything you open. First, the sample pack and the per-recording sample outputs are described separately. The outputs page shows a head-mounted Insta360 X5 recording at 4608 × 4608 and 30 fps, with an IMU at about 1,000 samples per second. The instrumented rig is stated separately as a head-mounted 4K/60 camera, two wrist cameras, an active-stereo depth sensor and 200 Hz IMUs. The site does not map one onto the other, so ask which capture setup your sample came from. Second, the web clips on the outputs page are downscaled to 1080 × 1080 for the browser, while delivered files are the full 4608 × 4608 frame at the original length. Do not judge resolution from a web preview.

Two references apply, and it helps not to mix them. For the sample pack, the site states 8 synchronized streams per episode but does not list which eight, so the exact stream list for a given sample is agreed per request. For field names, dtypes and units, the episode schema page is the reference, and the site says the sample pack ships in that schema. Check what is present against both your own stream list and the schema, and record anything missing as a finding rather than assuming it will arrive later.

The schema page also states that streams are stored at native rate and resampled by the loader, never in the delivery. So a depth array with fewer frames than the head camera is expected, not a defect. What would be a finding is a stream that has no timestamp array of its own, because the independent sync check in the next section depends on it. If the shapes in your sample differ from the example layout, ask whether that reflects a different capture setup. The site does not state a mapping between the 4608 × 4608 per-recording outputs and the rig shapes above, so a difference is a question rather than a fault.

If you received the per-recording files instead of the full pack, the checks are simpler and the contents are different. The outputs page lists eight files, and none of them is a depth or hand-pose file. The hand and body joints, masks and action segments are listed in annotations.json, which the site places in the full pack. So do not read the absence of hand pose from a per-recording folder as a gap in the data. Read it as a sign that you are looking at the video, IMU, calibration and privacy layer only.

This is the check that most separates a stated claim from a measured one. The site states the design: all streams are hardware-triggered onto a single PTP (IEEE 1588) clock domain, with int64 nanosecond timestamps per stream in that domain so that alignment can be verified independently. The stated residual is a median of 1.12 ms and a p99 of 1.94 ms against a 2.0 ms reject ceiling. The sample request page tells you how to test it: because every stream carries nanosecond timestamps, compute the offset between the head, wrist, depth and IMU streams yourself and compare it with the stated median and ceiling.

One workable approach is to pair each frame or sample in one stream with the nearest timestamp in another, then look at the distribution of those offsets across an episode. Read the result against the stated figures rather than against a number of your own. Those figures describe production residuals, so a small sample is a spot check and will not reproduce a p99 on its own. The site’s stated reject rule is that an episode is rejected for a breach beyond the ceiling lasting more than three consecutive frames, and that an episode exceeding the ceiling is discarded at capture rather than shipped. If what you find sits outside those stated rules, treat it as a specific question for the provider, not a verdict.

For the per-recording outputs, the checks are lighter. The imu.csv file is timestamped in microseconds on the same timeline as the video, and the site states about 1,000 samples per second, so a 10-second clip should carry roughly 10,000 rows. That is a quick plausibility test on row count and time column. The camera_dump.txt file carries per-lens fisheye calibration and per-frame exposure data. The outputs page does not state a reprojection figure for it, so if you need one, ask. Do not infer one from the pack-level figure.

The site states per-frame hand joints as 21 joints per hand in 3D, delivered as hand_joints with shape [T, 2, 21, 3] in metres in the head frame, alongside hand_visible, a per-joint visibility flag. It also lists body joints, instance masks, contact events and action segments, each timestamped and aligned to every other stream. The sample request page gives the review method: overlay hand joints, masks and action segments on the video and look at occlusion, drift and boundary placement. The full pack includes annotations.json, with hand and body joints, masks, action segments and calibration, and an episode.rrd Rerun recording with every stream on one timeline, which makes the visual pass quick.

Choose frames by situation rather than at random. Look at the moment of grasp, the moment of release, and any frame where one hand passes behind an object or the other hand. Those are the frames where hand pose is hardest and where a label error would hurt a policy most. Step through an action segment boundary frame by frame and see whether the verb and noun describe what you see. For masks, look at the edge of the object while the hand is closing on it.

Expect no accuracy figure to compare against. The site states a second-reviewer pass on labels in production QA, and states that every recording in the per-recording outputs is human-reviewed for task match, visible hands and no staged or idle footage. It does not state a label accuracy number, so your own overlay review is the evidence, and any figure a provider quotes should come with the document behind it. If your task needs a label that is not in the schema list above, that is a spec item. Name it in the written spec rather than hoping the next batch carries it.

The site states the anonymization design plainly. Faces, license plates and screens are detected, tracked and irreversibly blurred, spoken identifiers are scrubbed from audio, and a human reviewer confirms the result. The pass runs on every episode at ingest, there is no separate un-redacted master, and there is no step for you to run yourself. A batch data card records the detector version, pass timestamp and reviewer confirmation per episode, which is what makes the privacy claim checkable rather than asserted.

In a sample, the practical check is to look. Scrub through the video for unblurred faces, plates and screens, paying attention to the edges of the frame and to people who enter late. For the per-recording outputs, the site says faces are detected on every frame and delivered blurred or as per-frame face bounding boxes, and the clips shown on the web have faces blurred. If your sample arrives with boxes rather than blurred pixels, ask which applies and who applies the boxes. Also check that file names and metadata carry no personal identifiers: the site states a human-readable, PII-free layout of project, category, activity, slot and version.

Consent is the part a file cannot show you, so it is a question of records. The site states that every contributor signs a release before capture, that the same standard covers anyone who appears in frame incidentally, and that the consent chain is documented and warranted by the provider. For a custom collection, the site describes the consent artefacts for the batch and a data card listing jurisdictions of capture. Whether the free sample pack itself includes a data card or consent records is not stated, so ask rather than assume.

Read the license before you rely on anything. The site’s terms state that sample datasets are provided for evaluation only. Redistribution, resale or use of sample data in a production or commercially deployed model is not permitted unless a separate commercial license has been signed. Treat the sample as a test fixture, not a training corpus. A custom collection is different: the site states a perpetual, irrevocable, worldwide, buyer-owned license, non-exclusive by default, with exclusivity available when requested at the specification stage. Do not plan around one on the strength of the other.

A sample also has hard limits on what it can show. It is existing footage captured for a different purpose. It cannot show your task, your environments or your condition cells. It cannot show how many episodes would be rejected against your reject rule, how coverage would fill against your spec, or how long your environments would take. The site makes the same split: the free sample pack is enough to sanity-check the loader and label quality, and a Spec Pilot is enough to move a policy. Writing the limits down at the end of the review is what lets the sample feed the next step instead of ending in a vague impression.

The sample is one step in a sequence the site states. In practice the order is sample, then a written skill spec, then a Spec Pilot, then a quote and scale, though a team that already trusts the schema can go straight to a quote. A Spec Pilot is a small, fast batch captured fresh against your written spec, put through the same reject log and coverage reporting as a full order, billed per validated hour and held to the same license as a full order. A quote is a scoped estimate against your written spec, stated as arriving within one business day of the intake form. The site says rejected episodes, recaptures, calibration time and travel are never billed.

The stated timelines help you plan the step after the review. A spec conversation happens the same week. The timeline then runs from a signed spec: where contributor coverage is deep, such as home kitchens, retail handling and light assembly, first validated data arrives in a median of 48 hours, and where coverage is thin, such as construction, clinical, agriculture and hospitality, it is typically three to four weeks. Time to full volume is not stated as a single figure, and your own internal approvals sit outside these numbers. Once a pilot returns, the pilot evaluation guide explains how to judge it against the spec, and the scaling guide covers the move to production volume.

Send the questions in one message, with your specific findings attached. A good answer is specific, written and checkable. A vague answer is the signal to ask for the document behind it. The questions below follow the review order, so each one maps to a check you have just run.

Ready to move past the sample? Write the spec, then request a custom dataset and describe the tasks, environments, streams and format you need. A Spec Pilot or a quote is scoped against that written spec.