Automotive Service Robotics Training Data

A robot that torques a wheel, drains fluid, removes trim, or reads a diagnostic connector needs to learn the same reach and grasp geometry a technician uses from that viewpoint, not a description of the task or a fixed-camera view of someone else doing it.

That means synchronized egocentric video, depth, 3D hand pose, and frame-accurate action segments, captured against a named list of service-bay skills rather than pulled from incidental garage footage never meant for manipulation training.

A custom automotive collection is scoped to the service-bay skills your policy needs: torque wrench to spec on a wheel, fluid drain and fill, trim and clip removal without damage, diagnostic connector seat and read, and filter swap in a constrained cavity.

You describe which of these tasks (and which bay setups — lift bay, tyre bay, engine bay work, under-vehicle on lift) your policy needs, and the collection is scoped to that list rather than a generic automotive category.

A head- and wrist-mounted camera keeps the torque, fluid, and trim motion in frame from the viewpoint a service policy will actually see from, even as the vehicle itself occludes parts of the scene from any fixed camera.

Depth, hand pose, and action segments are aligned to video on one hardware clock, so a policy fusing vision with depth or hand pose is training on streams that share the same timeline. Under-vehicle work is IR-hostile — painted and oily surfaces both absorb a depth projector pattern — so those episodes are flagged for depth quality rather than silently interpolated.

You describe the service tasks and bay setups your policy needs. We turn it into a written spec covering tasks, environments, capture tier, annotation depth, and accept criteria, then quote reach, timeline, and price against that document before any capture begins.

A Spec Pilot is a small, fast first batch that proves the spec and accept criteria before you scale volume on demand, running through the same seven-stage end-to-end pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization, and licensed delivery.

Each episode ships with 3D hand pose tracked through reach, grasp, and fit; instance segmentation; action segments; and grasp and contact events, alongside the synchronized video and depth streams, with depth-quality flags on episodes where under-vehicle conditions degrade the signal.

Delivery formats are RLDS, LeRobot, and WebDataset, plus HDF5, zarr, and Rerun .rrd, with readable converter source so you can retarget the schema to your own loader.

Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. Each episode links to a signed contributor release through a documented consent chain.

Because a service bay reliably contains license plates and VIN plates in frame, these are detected and irreversibly blurred at ingest without exception, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data.

Industrial robot pretraining data spans many sites, stations, and tools to give a foundation model a general prior before it is aligned to any one deployment; a custom automotive collection is the narrower, task-specific counterpart once that base layer exists.

Assembly task data shares the same bench-side, hand-pose-precision approach used here for wheel, fluid, and trim work ��� both are captured against a written spec at the same depth of annotation, just in different environments.