Egocentric Vehicle Diagnostics Data

It is real-world, first-person video of a person diagnosing a vehicle: connecting the scan tool, reading fault codes, inspecting components and identifying the fault. It is recorded from the operator viewpoint, with 3D hand pose, depth, contact events and frame-accurate action segments aligned on one hardware clock. The task is the one the site’s task catalog lists in a single row.

The data is human capture by paid, consented contributors, not robot or teleoperation data. This page is about vehicle diagnostics robot training data in the sense of what a policy can learn from watching skilled service work, and it makes no claim that any robot has been trained on it.

One, in the Auto Service Center & Tire Shop environment of the Automotive & Transport sector. It is named "Vehicle diagnostics" and its catalog description reads "Connect the scan tool, read fault codes, inspect components and identify the fault." It is rated Hard and its status is Active collection. The row lists a head-mounted phone or a multiview rig as the capture setup.

The site says nothing further about what Active collection means in volume or hours, so this page claims no completed collection and no delivered hours. It names only the steps and items that appear in the row: a scan tool, fault codes and components. The first table below reproduces the row as the catalog states it.

The egocentric vehicle maintenance page covers thirteen rows across oil and fluid, filter, battery, brake and diagnostics work. The diagnostics row is one of two in its diagnostics and general group, listed in a table alongside the rest. The egocentric car battery replacement page, the stereo oil and fluid change page and the stereo brake pad and rotor service page each take other rows of the same environment, and none lists a diagnostics row.

This page gives the diagnostics row its own treatment: the steps in order, how the capture works around a scan tool and a component, what each episode carries and where the failure modes are. The comparison table below sets the pages side by side.

Connecting comes first: the row names a scan tool. The data needs to show the tool, the hand that holds it and the point where it meets the vehicle, and a spec decides whether connecting is a step of its own. Reading fault codes follows, which is done on the tool itself, so the view of the scan tool matters more than the wide scene. Inspecting components is the largest hand and gaze motion in the chain, reaching to a part and looking closely at it.

Identifying the fault closes the sequence. The row states the step but the site does not say how it is labelled: the only language data is one task instruction per episode, so the fault that is found, the codes that were read and the vehicle they came from are not carried as labels unless the spec asks for them. The site states no fault code, scan tool brand, protocol or vehicle make for the row.

Across all steps, action segments carry the start and end of each one as the skill spec defines them. A spec can ask for the whole diagnosis in one episode or isolate a single step, and the segment boundaries are what let a policy learn the order.

Diagnostics is close-range work: a tool in one hand, a part in the other, and eyes moving between a screen and a component. A head-mounted camera keeps the whole reach in frame from the viewpoint a manipulation policy will see from, and two wrist-mounted cameras give a closer view of the hands and the part.

A multiview rig adds synchronized third-person angles when the spec asks for them. That is useful in a service bay, where the vehicle itself can hide the hands from the head view. Which angles a diagnostics order uses is set in the spec. The site does not state which part of the vehicle is inspected, so this page does not describe the vehicle layout.

The catalog lists one capture setup for the row: a head-mounted phone recording the first-person view of real work on site, with a multiview rig adding synchronized third-person angles when the spec asks for them. The wider stack the site documents for hard environments is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras and 200 Hz 6-DoF IMUs. Which of those a diagnostics order uses is Not stated, and the third table below sets what is stated beside what is not.

Streams are hardware-triggered onto one clock, synced under 2 ms. Depth is aligned to video, and distance-invalid pixels beyond the sensor reliable range are flagged rather than interpolated. The site does not state which shop, dealership or vehicle a collection runs in or on, and it does not state capture at a customer’s own facility, so the environment, access and schedule are agreed in the spec.

Each episode follows the same episode schema as every Firsthand custom collection: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events and frame-accurate action segments. The only language data is one task instruction per episode.

For diagnostics, hand pose tracks the operator through the grip on the scan tool and the reach to a component. Contact events mark when a hand takes hold of an item and when it lets go, and action segments mark the start and end of each step. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens. The fourth table below lists each item.

The scan tool screen is the first risk. Fault codes are read from a display, and the site states that screens are detected and irreversibly blurred at ingest. It does not say whether a scan tool display counts, so whether the codes stay legible in the delivered video is Not stated. Decide in the spec whether the reading matters before capture.

Occlusion is the second. The scan tool, a hand or the vehicle itself can hide the fingers and the part, and the vehicle can block the first-person view. The per-joint visibility flag is the answer the site gives, contact events mark when a hand meets an item, and a spec can ask for wrist or multiview angles so a grip the head camera loses is still recorded. Surfaces are the third: the site states that under-vehicle capture is hostile to infrared depth and that such episodes carry a depth-quality flag instead of being silently interpolated, but how depth behaves around a given component is Not stated, so check it on a Spec Pilot.

Sequence is the fourth. A diagnosis is a chain of dependent steps and the row does not say what one episode covers, so the spec decides whether one episode is one diagnosis or a longer visit. Scan tool brands, protocols, fault-code lists, vehicle makes, electric or high-voltage vehicles, accuracy claims, service times, throughput and any safety or certification claim are Not stated and are not claimed on this page.

Every batch is validated against signed accept criteria before it ships, with a reject log. The site states that an episode is rejected when the manipulated object is out of frame for more than 15 percent of it, when more than 35 percent of depth pixels in the manipulation region are invalid, or when an undocumented bystander appears.

Billing is per validated hour, and rejected items are recaptured at our cost. The site does not state an accuracy figure for diagnostics step labels, so what counts as a pass for a step boundary is set in the accept criteria.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The site also states that licence plates and VIN plates in service-bay footage are blurred at ingest without exception. The anonymization record is delivered with the data.

Whether shop signage, a work order or a customer’s documents are blurred beyond that general rule is Not stated, and this page makes no claim about it. Raise any such rule in the spec.

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets. None of them names vehicle diagnostics among its tasks. Vehicle diagnostics collection to a spec is therefore custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The site states no provider count or throughput for this row.

The off-the-shelf catalog shows what the capture rigs deliver, and the custom order applies the same approach to your task. The row is Active collection, and the site does not say how many hours exist for it.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are training-loader formats, HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a synchronized playback file for review. Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields.

Delivery is rolling, per validated batch, and formats are confirmed when the work is scoped.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The row is Active collection and is rated Hard, and the site does not say which speed applies to a diagnostics order, so the timeline is quoted against your spec before capture begins.

On-demand top-ups run against the same spec and accept criteria after each training run, so new diagnostics data stays consistent with what you already hold. The site does not state a batch size or a later-batch turnaround, so those are quoted against your spec.

Describe the steps you want and where each starts and ends: connecting the scan tool, reading fault codes, inspecting components and identifying the fault. Name the tools you expect to see, whether an episode covers one diagnosis or a longer visit, whether the scan tool display must stay legible, what counts as a reject and the variation your pipeline is missing. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and the accept criteria on real footage before you scale. The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program, under a perpetual, buyer-owned commercial license, with exclusivity available. The fifth table below shows what each spec element decides.