Now recruiting capture teams worldwide. Get paid to record first-person video of real work.

Apply to capture

Task family

Stereo Vehicle Diagnostics Data

Connect the scan tool, read the fault codes, inspect the components and identify the fault: a diagnosis is a chain of dependent steps where the answer is reached by looking, and a scan tool is a small target held close to a part. This page covers the stereo and container side of scan tool fault code robot data for the one catalog row that names this work: what a calibrated stereo pair adds to stereo vehicle diagnostics video, what MCAP is according to its own documentation, and what the site does not state. We run custom auto service data collection fast and on demand against a spec you write, as an automotive diagnosis dataset in MCAP, through custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Catalog row
Vehicle diagnostics
Status
Hard, Active collection, as the catalog lists it
Container
MCAP (.mcap), open source
License
Perpetual, buyer-owned

Why stereo and MCAP here

What do a stereo pair and an MCAP file add to vehicle diagnostics video?

01

Depth of the scan tool, the component and the hand

The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the scan tool, the part under inspection and the hands that hold and point at them sit in one frame.

02

Hands around a tool, flagged when hidden

A scan tool, a component or the vehicle itself can hide the fingers that hold them. 3D hand pose carries a per-joint visibility flag, so a hidden finger is flagged as hidden rather than silently guessed, and contact events mark when a hand takes hold of an item.

03

Connect, read, inspect and identify on one clock

The order matters because each step depends on the one before. Streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling, and action segments carry start, end, verb and noun as the skill spec defines them.

04

Several streams in one container

MCAP stores multiple channels of timestamped, pre-serialized data, so a stereo pair, hand pose and IMU can share one file on one log-time axis. Which streams go in the file is set in the skill spec.

Definition

What is stereo vehicle diagnostics data?

It is first-person video of a person diagnosing a vehicle: connecting the scan tool, reading fault codes, inspecting components and identifying the fault. It is recorded from their own point of view with two calibrated cameras, with hand pose and sensor streams alongside. The task is the one the site’s catalog lists in a single row. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The data is human capture by paid, consented contributors, not robot or teleoperation data, and this page makes no claim that any robot has been trained on it. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Catalog row

Which catalog row does this page rely on, and what is its status?

One, in the Auto Service Center & Tire Shop environment of the Automotive & Transport sector. It is named "Vehicle diagnostics" and its catalog description reads "Connect the scan tool, read fault codes, inspect components and identify the fault." Its skill is Hard and its status is Active collection. The row lists a head-mounted phone or a multiview rig as the capture setup.

Hard is the rating in the catalog’s Skill column, and the hard-task hub lists every row carrying it. The site says nothing further about what Active collection means in volume or hours, so this page claims no completed collection, no delivered hours and no delivered stereo footage. It names only the steps and items that appear in the row: a scan tool, fault codes and components. The first table below reproduces the row as the catalog states it.

How stereo applies

Is stereo already captured on the vehicle diagnostics row?

The site does not state that it is. The catalog lists a head-mounted phone or a multiview rig as the capture setup, so stereo applies here only through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot, which is a small first batch that proves the spec and accept criteria on real footage before you scale.

That distinction is the reason this page exists beside the egocentric one. The egocentric page treats the row as first-person capture, and this page treats the same row through the stereo offer and MCAP delivery. Neither page claims a stereo diagnostics collection already exists.

Difference

How is this different from the egocentric diagnostics page and the other auto service pages?

The egocentric vehicle diagnostics page covers the same row as first-person capture and lists RLDS, LeRobot, WebDataset, HDF5, zarr and Rerun .rrd as delivery formats. It does not centre a stereo pair or MCAP. The egocentric vehicle maintenance page covers thirteen rows grouped as oil and fluids, filters, battery, brakes, diagnostics and general, and the Vehicle diagnostics row is its Diagnostics group.

The stereo car battery, oil and fluid, and brake pad and rotor pages give stereo and MCAP treatment to other rows of the same environment, and none of them lists a diagnostics row. This page gives the diagnostics row its own stereo and MCAP treatment, stream map and failure modes. The comparison table below sets the pages side by side.

The steps

What are the steps in vehicle diagnostics, and what does each ask of the data?

Connecting comes first: the row names a scan tool. The data needs to show the tool, the hand that holds it and the point where it meets the vehicle, and a spec decides whether connecting is a step of its own. Reading fault codes follows, which is done on the tool itself, so the view of the scan tool matters more than the wide scene. Inspecting components is the largest hand and gaze motion in the chain, reaching to a part and looking closely at it.

Identifying the fault closes the sequence. The row states the step but the site does not say how it is labelled: the only language data is one task instruction per episode, so the fault that is found, the codes that were read and the vehicle they came from are not carried as labels unless the spec asks for them. The site states no fault code, scan tool brand, protocol or vehicle make for the row.

Action segments carry the start and end of each step as the skill spec defines them. A spec can ask for the whole diagnosis in one episode or isolate a single step, and the segment boundaries are what let a model learn the order.

Why stereo

Why capture vehicle diagnostics in stereo?

Diagnostics is close-range work: a tool in one hand, a part in the other, and eyes moving between a display and a component. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the scan tool, the part and the hands share a frame. That is the case the pair is for in stereo vehicle diagnostics video, where the question is how far a hand sits from a component and how it closes on a connector or a part.

The site also states that depth degrades on wet and reflective surfaces and handles that with per-frame confidence maps. The site gives no depth accuracy figure for diagnostics work, so whether a component is resolvable in depth is Not stated, and a spec should check it on a Spec Pilot.

Capture setup

What does the capture setup look like, and what is not stated?

The site describes two stereo setups and does not say which one a diagnostics order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a diagnostics order uses is Not stated, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. The catalog states that a multiview rig adds synchronized third-person angles when the spec asks for them, which matters in a service bay because the vehicle can hide the hands from the head view. The site does not state which shop, dealership or vehicle a collection runs in or on, so the environment, access and schedule are agreed in the spec.

MCAP

What is MCAP, per its own documentation?

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Streams

Which streams can ship in an MCAP delivery of vehicle diagnostics data?

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For diagnostics work the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Failure modes

What goes wrong when you record vehicle diagnostics in stereo?

Occlusion is the first risk. A scan tool, a hand or the vehicle itself can hide the fingers and the part, and the vehicle can block the first-person view. The visibility flag on each joint is the answer the site gives, contact events mark when a hand takes hold of an item, masks are recorded per frame, and a spec can ask for wrist views or a second angle so a grip the head camera loses is still recorded. The site does not say whether the scan tool or a component counts as the object for contact events, so settle it in the spec.

The scan tool display is the second. Fault codes are read from a display, and the site states that screens are detected and blurred at ingest without saying whether a scan tool display counts. Whether the codes stay legible in the delivered video is Not stated, so decide in the spec whether the reading matters before capture. Surfaces are the third: the site states that under-vehicle capture is hostile to infrared depth and that such episodes carry a depth-quality flag instead of being silently interpolated. How a stereo pair behaves around a given component is Not stated, so ask for the flag and the confidence maps to be delivered with each episode.

Sequence is the fourth. A diagnosis is a chain of dependent steps and the row does not say what one episode covers, so the spec decides whether one episode is one diagnosis or a longer visit. Calibration is the fifth, because stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it, when more than 35 percent of depth pixels in the manipulation region are invalid, or when an undocumented bystander appears, and a reject log ships with every batch. Scan tool brands, protocols, fault-code lists, vehicle makes, accuracy claims, service times, throughput and any safety or certification claim are Not stated and are not claimed on this page.

Privacy

What does the site say about people, plates and screens in frame?

For every delivered episode the site states that faces, plates and screens are blurred irreversibly, and that footage that cannot be de-identified is discarded. It also states that licence plates and VIN plates in service-bay footage are blurred at ingest without exception. Every episode links to a signed contributor release through a documented consent chain, and the anonymization record ships with the data.

Whether shop signage, a work order, a tool display or a customer’s documents are blurred beyond that general rule is Not stated, and this page makes no claim about it. Raise any such rule in the spec.

Scale

How do off-the-shelf coverage and custom collection combine for diagnostics work?

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. None of them names vehicle diagnostics among its tasks, and the one mixed-environment stereo set, S1, gives its breakdown by environment and task on request. So stereo vehicle diagnostics in MCAP is custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The site states no provider count or throughput for this row. The off-the-shelf stereo and hand-tracking video page shows what the rig delivers, and the custom order applies it to your task.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The row is Active collection, and the site does not say which speed applies to a stereo diagnostics order, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Ordering

How do you order a stereo vehicle diagnostics dataset?

Describe the steps and where each starts and ends: connecting the scan tool, reading fault codes, inspecting components and identifying the fault. Name the tools you expect to see, whether an episode covers one diagnosis or a longer visit, whether the scan tool display must stay legible, what counts as a reject and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot. Delivery is rolling, per validated batch, under a perpetual, buyer-owned commercial license, with exclusivity available. On-demand top-ups run against the same spec and accept criteria after each training run. The fifth table below shows what each spec element decides.

At a glance

What you get.

Catalog row
"Vehicle diagnostics" (Hard, Active collection), in the Auto Service Center & Tire Shop environment of the Automotive & Transport sector
Catalog description
"Connect the scan tool, read fault codes, inspect components and identify the fault."
Catalog capture setup
Phone (head-mounted) / multiview rig; a stereo pair is agreed in the skill spec
Stereo head rig (06)
Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAP
Hand pose
21 joints per hand with a per-joint visibility flag
Sync
Hardware-triggered onto one clock; 2.0 ms cross-stream ceiling
MCAP topics, schemas, rates
Not stated on the site; agreed in the skill spec
License
Perpetual, irrevocable, buyer-owned, non-exclusive by default

Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.

Catalog row

Which catalog row covers vehicle diagnostics?

The row relied on by this page, from lib/commercial-catalog-data.ts, with the catalog description, environment, skill, capture setup and status as listed.
Task (as listed)Description (as listed)EnvironmentSkillCapture setupStatus
Vehicle diagnosticsConnect the scan tool, read fault codes, inspect components and identify the fault.Auto Service Center & Tire ShopHardPhone (head-mounted) / multiview rigActive collection

Compared pages

How does this page differ from the neighbouring diagnostics, maintenance, battery, oil and brake pages?

Each page, what it covers and how it treats vehicle diagnostics. “Not covered” means the page leaves it out.
PageWhat it coversTreatment of vehicle diagnostics
This page: stereo vehicle diagnostics dataThe Vehicle diagnostics row in Auto Service Center & Tire ShopStereo pair and MCAP delivery, stream map and failure modes for this row only
Egocentric vehicle diagnostics dataThe same row as first-person captureThe steps in order, capture setup and failure modes; RLDS, LeRobot, WebDataset, HDF5, zarr and Rerun .rrd delivery; stereo and MCAP are not its focus
Egocentric vehicle maintenance dataThirteen rows grouped as oil and fluids, filters, battery, brakes, diagnostics and generalThe Vehicle diagnostics row listed in its Diagnostics group, as egocentric capture
Stereo car battery test and replacement dataThe three battery rows with a stereo pair and MCAP deliveryNot covered; these are battery rows, not diagnostics rows
Stereo oil and fluid change dataThe three oil and fluid rows with a stereo pair and MCAP deliveryNot covered; these are oil rows, not diagnostics rows
Stereo brake pad and rotor service dataThe three brake rows with a stereo pair and MCAP deliveryNot covered; these are brake rows, not diagnostics rows
Stereo tire rotation and pressure check dataThe tire rotation and pressure check row with a stereo pair and MCAP deliveryNot covered; this is a tire row, not a diagnostics row
Hard task data collectionEvery catalog row rated Hard, grouped by environmentThe hub row table lists Vehicle diagnostics and links here

Stated vs not stated

What the site states about stereo capture for vehicle diagnostics

Left to right: the item, what the site states, and what that means for a diagnostics order. “Not stated” means the site does not say.
ItemWhat the site statesFor a stereo diagnostics order
Catalog capture setupPhone (head-mounted) / multiview rig on the rowA stereo pair is a spec choice; not stated on the row
Stereo head rig (dataset 06)Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAPWhether a diagnostics order uses it: Not stated
Hard-environments stack4K/60 head camera, 120° lens; two 1080p/60 wrist cameras; depth 848×480 at 30 fps; 200 Hz 6-DoF IMUWhether a diagnostics order uses it: Not stated
Objects, hands and masksPer-frame instance masks stored at 848×480; per-joint visibility flag on hand pose; contact eventsStated; whether the scan tool or a component counts as the object is Not stated
Scan tool displayScreens are detected and blurred at ingestWhether a scan tool display counts, and whether fault codes stay legible: Not stated
Depth around a componentUnder-vehicle capture is hostile to infrared depth; episodes carry a depth-quality flag; stereo depth degrades on wet and reflective surfaces with per-frame confidence mapsBehavior of a stereo pair around a component: Not stated
CalibrationMeasured every session and shipped with the episode; 0.28 px RMS reprojection gateStated
SynchronizationHardware-triggered onto one clock; 2.0 ms cross-stream ceilingStated
Row statusHard, Active collectionNo volume, hours or delivered stereo footage is claimed on this page
Scan tool brands, protocols, fault-code lists, vehicle makesNot statedAgreed in the skill spec, or not claimed on this page
Accuracy claims, service times, throughput, safety or certification claimsNot statedNot claimed on this page
MCAP topics, schemas, ratesNot statedAgreed in the skill spec

Stream map

Streams for diagnostics work and what the site states about an MCAP file

Left to right: the stream or item, what the site states about it, and what the site states about it in MCAP. “Not stated” means the site does not say.
Stream or itemWhat the site statesIn an MCAP delivery, as the site states it
Stereo pairDual fisheye, 1080p per eye, 62 mm baseline (dataset 06)Dataset 06 is stated as shipped as MCAP; channel layout not stated
Depth with confidence mapsRecovered from the calibrated stereo pair; confidence map masks low-confidence regionsNot stated
Hand pose21 joints per hand with a per-joint visibility flagChannel layout and joint order not stated
IMU9-axis (dataset 06); 6-DoF at 200 Hz (hard-environments stack)Channel layout and rate not stated
Contact events, action segments, instance masksListed in the site’s episode schema; masks stored at 848×480Not stated
Task instructionOne per episodeNot stated
CalibrationMeasured every session and shipped with the episodePlacement in the file not stated
Topic names, schemas, encoding, ratesNot statedAgreed in the skill spec

Skill spec

What the skill spec fixes for a stereo vehicle diagnostics order

Spec element and where a buyer can see it in the delivery. “Set per program” means the site states no fixed value.
Spec elementWhat it decidesWhere it shows up
Tasks and orderWhat one step is: connect, read, inspect or identifyEpisode boundaries and action segments; set per program
Episode scopeOne diagnosis or a longer visitSet per program; the site does not state an episode length for this row
Rig and streamsStereo pair, wrist views, a second angle, hand pose, IMU and any othersThe channels you can read; confirm on the Spec Pilot
Object definitionWhether the scan tool or a component counts as the objectContact events; set per program
Display legibilityWhether the scan tool display must stay legibleBlurring of screens in the delivery; set per program
Step labelsWhether connect, read, inspect and identify are separate labelled stepsAction segments; set per program
MCAP topics and encodingWhat your reader expectsSet per program
Calibration deliveryInside the file or beside itSet per program
Delivery formatHow the data is packagedMCAP alone, or beside RLDS, LeRobot, WebDataset, HDF5 or zarr
Accept criteriaWhat passes QA, including what counts as a rejectReject log per batch
PrivacyHow people, plates and screens in frame are protectedFaces, plates and screens blurred at ingest
LicenseWho owns the datasetPerpetual, irrevocable, buyer-owned, non-exclusive by default

Example footage

What does egocentric stereo capture look like?

One real clip already published on this site, shown as a format and rig example: a head-mounted stereo capture with hands in frame. It does not show vehicle diagnostics.

egocentric_stereo_sample.mp4 · head-mounted stereo · hands in frame · 1920 × 1080 web preview · rig example, not a diagnostics clip

No vehicle diagnostics clip is published on the site, so this is labelled as a format and rig example. Request a sample to see delivered files. See stereo sample page, stereo + wrist-cam example videos, request an egocentric data sample.

FAQ

Stereo vehicle diagnostics video, answered.

What is stereo vehicle diagnostics data?
First-person video from two calibrated cameras of a person connecting a scan tool, reading fault codes, inspecting components and identifying the fault, with depth and hand pose alongside.
Which catalog row covers it, and what is its status?
One row, "Vehicle diagnostics", in the Auto Service Center & Tire Shop environment: "Connect the scan tool, read fault codes, inspect components and identify the fault." It is rated Hard and its status is Active collection. This page claims no volume, hours or delivered stereo footage.
Is stereo already captured on this row?
Not stated. The catalog lists a head-mounted phone or multiview rig, and stereo applies only through the site’s general stereo offer: a stereo pair is agreed in the skill spec and confirmed on a Spec Pilot.
Is there an off-the-shelf automotive diagnosis dataset?
The site lists none. Stereo vehicle diagnostics in MCAP is custom collection, at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Can it be delivered as MCAP, and what is in the file?
Yes. Firsthand offers MCAP delivery. Topic names, message schemas and rates are Not stated on the site and are agreed in the skill spec.
Will the fault codes stay legible in the video?
Not stated. The site says screens are detected and blurred at ingest and does not say whether a scan tool display counts, so decide in the spec whether the reading must stay legible.
Does the data include scan tool brands, fault codes or vehicle makes?
Not stated. The site lists no scan tool brand, protocol, fault-code list, vehicle make, accuracy claim, service time, throughput, safety or certification claim for the row, so ask for any such label in the spec.
How fast can a custom diagnostics dataset be collected?
Collection is fast and on demand to a written spec, and the timeline is quoted against your spec before capture begins. Most programs start with a Spec Pilot.

Sources

Where is MCAP documented?

Request a custom stereo vehicle diagnostics dataset.

Tell us the steps and where each starts and ends, the tools you expect to see, whether an episode covers one diagnosis or a longer visit, whether the scan tool display must stay legible, which streams you want in the MCAP file and which angles you want beyond the stereo pair. We will quote reach, timeline and price against a written spec.