Egocentric Grasping Data Collection

It is first-person (egocentric) video of a human grasping an object, recorded from a head-mounted camera at the operating viewpoint a manipulation policy will actually see from. The grasp itself — the moment fingers make contact with and close around an object — is the focal event, but it is captured inside a continuous episode, not as an isolated clip.

This page is about buying that data to a written spec, not a general hand-object interaction page, a pick-and-place page, a hand-tracking page, or a contact-event definition; those are linked below and compared in the table further down this page.

The data is human demonstration recorded from the performer’s own viewpoint. This page makes no claim that robot or teleoperation grasps are captured.

A grasp is a short, high-precision event. First-person capture puts the camera at the viewpoint a manipulation policy itself will operate from, so the geometry a policy learns to condition on at inference is the same geometry it trained on, with the hands and the object in frame.

A policy that only ever sees a grasp already completed, or sees it from across a room, has little signal for how fingers actually approach, close around, and release an object. Grasp-focused egocentric data keeps the hand and the object in frame through that whole sequence, with the moment of contact marked on the same clock as the rest of the episode.

This page does not state a grasp success rate, a grasp-quality metric, or a benchmark number, because none of those are fields this site delivers; what is delivered is described in the tables below.

The skill spec names the objects, grasping tasks, and settings before capture starts. Each named combination of object, task, and setting becomes a coverage cell with validated hours tracked against it, the same way any other spec line item is tracked. The spec describes what the contributor is asked to do, not a grasp-type label attached to the footage.

Firsthand designs the first draft of a skill spec with you on a call, including hours budgeted per object and task cell. A Spec Pilot, a small and fast first batch, is reviewed against every axis of the spec before volume scales, with the same reject log and validated-hour billing as a full collection.

The spec is also where failure-case coverage for grasping is planned, if it is wanted: an occluded grasp, a slipped grasp that is re-grasped, or a grasp attempted in dim light are named scenarios like any other, not an afterthought.

A head-mounted camera keeps the hand and the object in frame from the operating viewpoint through the whole reach-grasp-release cycle. An optional wrist-mounted camera can run alongside it for a closer-range view of the grasp itself. Where manipulation precision requirements are high, a stereo pair adds binocular depth for finer contact-level geometry.

Head and wrist IMU streams are synchronized to the same clock as the cameras. All streams are hardware-triggered from a single PTP grandmaster clock and aligned under 2 ms, with per-stream nanosecond timestamps in the PTP domain, so the frame where fingers make contact lines up exactly with the depth and hand-pose frame for that same instant.

Capture takes place in the settings this site already lists for custom collections, not at a customer’s own facility.

A grasp is not a single tag on a clip. It is described by several streams recorded on one clock, each covering a different part of the event. The table below lists each element and the format field that carries it.

hand_joints tracks 21 joints per hand in 3D, in metres, in the head-camera frame, on every frame of the reach, grasp, and release. hand_visible flags occlusion per joint, which matters most during the grasp itself, when fingers are most likely to be hidden from the camera by the object or the other hand. contact_events marks when and where contact is made and broken. action_segments gives the grasp its place inside the larger task, as a {start_ns, end_ns, verb, noun} record on the same clock.

The fields documented on this site do not include a grasp-quality score, a grasp-type taxonomy, a 6-DoF grasp pose, an object pose, or force, pressure, or tactile channels, and this page makes no claim to any of them. What is captured is the hand’s own tracked pose and the timing of contact, not a model of the grasp’s mechanics.

Acceptance is measured against signed accept criteria written into the spec before capture begins. Each batch goes through per-batch QA and a reject log ships with every batch. An episode is accepted only if RMS reprojection stays under 0.28 px and the drift is within tolerance; otherwise it goes to the reject log and is recaptured at our cost.

Coverage against the named object and task cells in the spec is checked the same way: a cell either has enough validated hours against it or it does not, checked against the list rather than an impression of variety.

Only footage that clears the criteria is billed as a validated hour.

Every contributor signs a documented consent release, and each episode links to that signed release through a documented consent chain. Faces, license plates, and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships.

Every custom collection ships under a perpetual, buyer-owned commercial license. Exclusivity is available so the data is never resold. This page makes no certification claims.

Delivery formats are RLDS, LeRobot, and WebDataset, plus HDF5, zarr, and Rerun .rrd for full-fidelity review of the aligned streams. The episode schema is the canonical layout, and each format is a projection of it, so hand_joints, contact_events, and action_segments for a grasp are part of the layout your loader reads. In WebDataset, each sample’s ann.json carries hand_joints, action, and contact_events. Converter source is readable, so you can retarget the schema to your own loader.

Units are metric throughout: depth in millimetres, hand joints in metres.

Describe the objects, grasping tasks, and settings your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth, and accept criteria, then quote reach, timeline, and price against that document before any capture begins. Pricing details are on the pricing page rather than here.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and that timeline is quoted up front. On-demand top-ups run against the same spec and accept criteria, so new grasping data stays consistent with what you already hold.