Egocentric Video of Real Workplace Tasks

Egocentric video of real workplace tasks is first-person footage of people doing actual job tasks — picking and packing in a live warehouse, facing and restocking a retail floor, diagnosing and servicing a vehicle in a working bay, assembling or fastening at a bench, prepping food in a working kitchen — captured on head- and wrist-mounted cameras in the real setting where the work happens, against a written skill spec.

It is distinct from a staged recreation of the same task in a studio or lab. The setting is a working site, not a set built to look like one, and the people performing the task are doing their actual job rather than acting it out for the camera. Consent is documented and bystanders who did not opt into being recorded are de-identified, the same as for any other custom collection on this site.

A model that will eventually act in a workplace needs training data that reflects the clutter, pace, and interruptions of that workplace — not a cleaner, slower lab recreation where every object is placed for the camera and every motion is deliberate. A staged recreation tends to smooth over exactly the small recoveries, reaches, and hesitations a worker makes without thinking about them, which are often the detail a policy needs to generalize.

Egocentric capture adds a second advantage on top of the real setting: it records the first-person reach geometry — where the hands are relative to the head, how the body orients toward an object before a grasp — that a fixed overhead or third-person camera does not capture. Combined with a real working site instead of a staged one, that gives a training pipeline both the correct viewpoint and the correct context for the task.

The settings captured are warehouse picking, retail shelf stocking, automotive service bay work, tool use and assembly, and kitchen food preparation. Each setting has its own dedicated page on this site with example tasks and dataset detail: warehouse picking covers picking, scanning, packing, and pallet-building; retail covers facing, restocking, price changes, and planogram resets; the automotive service bay covers diagnostic, wheel, fluid, and trim work; tool use and assembly covers bench assembly, fastening, measuring, and fitting; and kitchen covers two-handed food preparation in a real domestic kitchen.

A buyer is not limited to ordering one setting at a time. Because every setting is specified, captured, and QA’d through the same written skill spec and process, a single order can combine tasks across settings — for example, warehouse picking alongside bench assembly — as long as each task is written into the spec the same way.

A workplace task is written into the same skill spec format used for every custom collection on this site: the setting, the tasks to demonstrate, the scenes and objects involved, which streams to capture, which labels the training pipeline needs, and the delivery format the pipeline expects. A warehouse pick-and-pack task and a bench assembly task are specified the same way — only the setting, tasks, and objects in the spec change.

Writing the spec before capture begins means the accept criteria — what counts as a correctly completed demonstration of the task, what streams and labels are required — are agreed before footage exists, rather than negotiated after the fact against footage that may not meet them.

Each workplace episode captures the same stream set the spec calls for across any setting: synced head and wrist RGB for the first-person view of the task from the worker’s viewpoint, depth derived from the calibrated stereo rig for scene geometry, 6-DoF IMU on the hardware-triggered PTP clock for motion, 21-joint 3D hand pose with per-joint visibility, and frame-accurate action segments marking the start and end of each labeled action.

RGB is the baseline stream for every episode; depth, IMU, and hand pose are added where the spec calls for them, the same as for any other custom egocentric collection. Which streams and labels a given workplace task needs is decided in the spec, not assumed — a simple facing task may need less than a multi-step diagnostic task.

A workplace has bystanders — coworkers, customers, passersby — who did not opt into being recorded. Every bystander face and license plate is blurred at ingest as part of the same anonymization process used across this site, and clips where a bystander cannot be de-identified are discarded rather than delivered with a quality caveat.

Consent from the person performing the demonstrated task is documented as part of the capture process, the same as for any other custom collection. Because de-identification discards clips rather than patching them, yield varies by setting and by how busy a given site is, rather than following a single fixed rate across every workplace.

Every episode passes through calibration and synchronization checks on every active stream, plus label and annotation review for hand pose and action segments where those are included, before it is delivered. Episodes that fail these checks are rejected rather than shipped with a caveat, the same acceptance process used for every other custom collection on this site.

Accepted episodes ship in RLDS, LeRobot, WebDataset, HDF5/zarr, or Rerun .rrd format — whichever the spec calls for — under a perpetual, non-exclusive, buyer-owned license by default, so the data can go straight into a training pipeline without renegotiating rights after delivery.

Start with a written skill spec for the setting and tasks needed — one setting or several — covering the scenes, objects, streams, labels, and delivery format the training pipeline expects. That spec is the same starting point used for a Spec Pilot, a small, fast first batch that proves the spec against real footage before a buyer commits to the full volume.

Running a pilot before scaling to the full collection means the reject rate, coverage, and stream quality a buyer would get at scale are measured on a small batch first, while the cost of correcting a spec gap — an ambiguous task definition, a label that needs tighter guidance — is still small.