Egocentric Heavy Machinery Operation Data

Egocentric heavy machinery operation data is first-person video of a person operating machine controls, recorded from their own point of view with synchronized sensor streams alongside. On this site the task is the one in a single catalog row: operating the levers and control panels of presses, cranes or machinery from the operator station.

This page is about buying a custom collection of those demonstrations. It is narrower than a general manufacturing collection and than a welding or metal fabrication collection. The subject here is the control-operating step itself, how it is specified and how it is labeled so a training pipeline can use it.

One row counts on this page: "Operating Heavy Machinery Controls". It sits in the Heavy Industry & Fabrication environment, in the Manufacturing & Workshops sector, and the catalog lists the environment examples as metal fabrication, woodworking, glass manufacturing, a concrete plant and a welding workshop.

The row is described as "Operate levers and control panels of presses, cranes or machinery from the operator station." It is rated hard, lists the capture setup "Phone (head-mounted) / multiview rig", and is marked available on request, not ready to deploy. The first table below reproduces it as the catalog states it.

The description names three kinds of machine, presses, cranes and machinery, in one row. The site does not list a separate row for each, so a spec that needs only one of them is settled in a Spec Pilot rather than assumed from the catalog.

Heavy Industry & Fabrication holds five rows, all rated hard and all available on request. Only one of them is a machinery-controls row. The other four are concrete work, glass manufacturing, metal fabrication and welding, and the second table below lists each with the reason it is left out.

Metal fabrication does mention shop machinery, but its description is to cut, bend, drill and weld metal parts into assemblies, which is a fabrication task rather than operating a control panel. Sub-tasks that are not the listed row, such as driving a vehicle, signalling a load or running a production line from a control room, are not claimed here and are settled in a Spec Pilot.

Operating a control panel is a hand task, and the hands spend much of it behind or over the controls. Fingers disappear behind objects in manipulation, which is why 3D hand pose carries a per-joint visibility flag. Where a lever is gripped, the fingers wrap around it and part of the grip is out of view.

The site does not describe the operator station itself. It does not say whether the station is on a live machine, a trainer or a mock-up, how many controls it has, or how a control is labeled, and it gives no figure for hand pose accuracy at a control panel. Those details are agreed in the spec.

Each attempt is delivered as one episode, and a spec needs a decision up front about what one attempt is: one lever action, one panel sequence, or one full operating pass from the operator station. The site does not set that length, so the skill spec does.

Inside the episode, the synchronized streams run from start to end and action segments divide it into labeled spans. A segment might be one control action as the spec names it. This is an illustration of how a spec can cut an episode, not a stated duration.

The catalog lists the capture setup for this row as a head-mounted phone or a multiview rig. The instrumented tier is a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors.

The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The site does not say which of those streams the phone mode carries for this row. The site does not state a camera count, resolution or frame rate specific to machinery work, and no robot or teleoperation data is captured.

Inside each episode, action segments divide the recording into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets an item, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.

Hand pose is delivered with 21 joints per hand and a per-joint visibility flag for occluded fingers. That is the label that carries how the fingers grip a lever or press a control. The site does not say whether a lever or a panel control counts as the item in a contact event, so the spec is the place to settle it.

The site does not list a label for the machine itself: no control position, machine state, load, lever travel or completion score, and no contact force reading.

From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. The site does not say which timeline applies to this environment, so ask when you request the quote.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. Collections are billed per validated hour, and the site states no price list or minimum order for this family. Capture at your own facility is listed as not offered.

Describe the control task, the kind of machine, the setting and the variation your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first.