Vehicle maintenance

Egocentric Vehicle Maintenance Video Data

A maintenance job is a short chain of exact hand steps: drain, swap, refill, test, tighten. We collect custom egocentric video of the vehicle maintenance tasks this site lists, on demand, to a written skill spec, with synchronized streams, 3D hand pose, action segments, anonymization and a buyer-owned license.
Listed maintenance rows
Thirteen, in one auto service setting
Fluid, part and torque details
Not stated
First batch
Median 48 h in deep-coverage settings
License
Perpetual, buyer-owned

Requirements

What does maintenance work need that a generic garage clip does not?

01

Each step kept, in order

A useful clip keeps the whole job in one episode, with action segments so the drain, the swap and the refill can each be sliced out.

02

A clear line between what is listed and what is not

The catalog lists no wiper, bulb or multi-point inspection row, and no label for fluid type, part type or torque value. This page marks those items "Not stated".

Definition

What is egocentric vehicle maintenance data?

It is first-person video of a person servicing a vehicle: draining and refilling oil, swapping a filter or battery, replacing brake pads, or connecting a scan tool and reading fault codes. The camera is head-mounted, with two wrist cameras for a closer view of the hands. The contributor is a paid, consented person following a written skill spec, not an actor and not a robot.

Where the site states nothing, a table below says "Not stated".

Task coverage

Which maintenance tasks can be captured?

The task catalog already names maintenance work. Thirteen rows sit in one setting, Auto Service Center & Tire Shop, in the Automotive & Transport sector, whose examples read "Car garage, dealership service bay, oil change, tire rotation, brake work, vehicle diagnostics." The table below reproduces the rows as the catalog states them.

They fall into five groups: three oil and fluid rows, two filter rows, three battery rows, three brake rows, and two diagnostics and general rows. One is listed as easy, seven as medium and five as hard. All use the capture setup "Phone (head-mounted) / multiview rig".

Four rows give a setting, "Automotive repair shop with vehicle lifts, tool chests, and dedicated service bays", where the other rows give steps. They are reproduced as listed; the site says nothing more about how lifts are used. Three brake rows also mention removing a wheel, so they overlap the tire and wheel service page.

Tire and wheel rows and washing and detailing rows sit in the same catalog but have their own pages, so they are left out. The catalog lists no row for replacing wipers or bulbs and none for a multi-point inspection.

Episode contents

What does each maintenance episode contain?

An episode is one contributor performing one task from the skill spec, such as one oil change or one battery swap. The wider stack is ten streams, including a head-mounted global-shutter 4K/60 camera with a 120 degree lens, two wrist-mounted 1080p/60 cameras, depth, 200 Hz 6-DoF IMUs, 3D hand and body pose, instance segmentation and action labels, hardware-aligned within the 2 ms ceiling the site documents.

Every episode carries int64 nanosecond timestamps per stream, camera intrinsics and extrinsics, hand joints for both hands, instance masks, action segments and contact events. The rig is calibrated at the start and end of every session, and an episode is accepted only if RMS reprojection stays under 0.28 px.

Labeling and delivery

How is vehicle maintenance data labeled and delivered?

The unit of labeling is the action segment: a taxonomy of 97 verbs and 512 nouns, annotated start and end to the frame and human-reviewed. A spec that says "change the oil" gives one long segment. A spec that names drain, swap the filter and refill gives a sequence you can slice by step.

Data is delivered in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, with converters shipped as readable source. You receive a perpetual, irrevocable, buyer-owned commercial license, the consent artefacts for the batch and a data card that lists jurisdictions of capture. Billing is per validated hour, and rejected episodes and recaptures are never billed.

Robot learning

Why is vehicle maintenance hard for robot learning?

The site states no maintenance learning result, so this is plain reasoning about the task, not a finding. A job is a chain of dependent steps, and the order matters: the filter comes off after the drain, and the terminals are tightened after the swap.

Two consequences show up in the data. Sequence: action segments keep each step in order and sliceable. Occlusion: the site’s automotive service page says the vehicle itself frequently occludes the view, and that under-vehicle work happens in tight, low-light cavities where depth degrades, so those episodes are flagged for depth quality instead of silently interpolated.

Firsthand captures people, not robots. There is no robot joint-angle log, robot result, benchmark or teleoperation data, and using the episodes for a specific robot means your own retargeting step. The standard capture lists no force or tactile stream, only contact events. A separate glove tier has its own page, and this page makes no claim about it for maintenance work.

Privacy and settings

Where is it captured and how are bystanders, customers and license plates protected?

The catalog places every maintenance row in Auto Service Center & Tire Shop. The site’s automotive service page lists bay setups as lift bay, tyre bay, engine bay work and under-vehicle on lift. It does not state which sites or countries host capture, or how many, and it states no capture at a buyer’s own garage or facility. The spec settles location and access.

Only what the site already states applies to privacy. Every participant signs a written release granting commercial training rights before capture begins. Faces and license plates are detected and irreversibly blurred, and the automotive service page adds VIN plates, blurred at ingest without exception. A human reviewer confirms that no un-consented person is identifiable in any frame, audio is scrubbed of identifying speech, and a clip where a bystander cannot be de-identified is discarded. The site says nothing separate about customers or vehicle owners. This is not legal advice, and the executed agreement controls.

Speed

How fast can a custom vehicle maintenance dataset be collected?

The site states one published timeline: from a signed spec to a first validated batch, a median of 48 hours where coverage is already deep, and typically three to four weeks where coverage is thin because operator vetting is the bottleneck.

Four of the thirteen rows are marked ready to deploy, meaning contributors and site access are set up and capture starts once a spec is signed. The other nine are marked active collection, meaning recording is running now, so a spec can join an existing capture stream. The site does not say which timeline applies to which row, so ask when you request the quote.

Getting started

How do you start a pilot?

Start with the free, ungated sample pack to check the loader and the label quality, then move to a Spec Pilot: a small first batch against the written spec that proves the fill rate and the reject rate on your conditions before you scale.

A good brief names the maintenance task, the order of steps, how each step starts and ends, and what counts as a reject. The site’s skill-spec guide says to name the verb and the noun rather than a broad activity. Then request a custom dataset.

Capture setup

What is the capture setup for a vehicle maintenance collection?

The capture setup as stated on this site, applied to maintenance tasks. Counts of contributors and hours are set in the skill spec.
PartAs stated on this site
Head cameraGlobal-shutter 4K at 60 fps with a 120 degree lens
Wrist camerasTwo wrist-mounted cameras at 1080p and 60 fps
IMU6-DoF at 200 Hz
SyncHardware-triggered onto a single PTP clock domain, within the 2 ms ceiling
CalibrationRe-solved at the start and end of every session; RMS reprojection under 0.28 px
Catalog capture setup, all thirteen rows"Phone (head-mounted) / multiview rig"
Capture setup for wipers, bulbs or a multi-point inspectionNot stated

Every collection runs through the same seven-stage end-to-end custom collection pipeline.

Task catalog

Which maintenance rows does the catalog name?

The thirteen non-tire, non-washing rows in Auto Service Center & Tire Shop, reproduced as the catalog states them. All use "Phone (head-mounted) / multiview rig". Where the description column gives the setting instead of steps, that is how the catalog lists it. No row names wipers, bulbs or a multi-point inspection.
Task (as listed)Description (as listed)GroupSkill levelStatus
Oil & Fluid ChangeDrain engine oil, replace the filter, refill oil and top up coolant and other fluids.Oil and fluidsMediumReady to deploy
Drain and replace engine oil using a drain pan and filter wrenchDrain oil into the pan, swap the filter with a filter wrench and refill to level.Oil and fluidsMediumActive collection
Drain, replace, and refill engine oil and oil filter using a wrench, drain pan, and funnel.Automotive repair shop with vehicle lifts, tool chests, and dedicated service bays.Oil and fluidsMediumActive collection
Filter & Component ReplacementRemove and replace air, cabin and oil filters and worn components with hand tools.FiltersMediumReady to deploy
Inspect, remove, and install air and cabin filters using a screwdriver and a shop rag.Automotive repair shop with vehicle lifts, tool chests, and dedicated service bays.FiltersEasyActive collection
Battery Test & ReplacementTest the battery with a multimeter, disconnect and swap it and clean the terminals.BatteryMediumReady to deploy
Test and replace a car battery using a multimeter and socket setRead voltage with a multimeter, unbolt the old battery, fit the new one and tighten terminals.BatteryMediumActive collection
Test, remove, and replace car batteries and terminals using a wrench and a multimeter.Automotive repair shop with vehicle lifts, tool chests, and dedicated service bays.BatteryMediumActive collection
Brake Pad & Rotor ServiceLift the car, remove the wheel and caliper, fit new pads and rotor and torque the bolts.BrakesHardReady to deploy
Remove and replace brake pads using a jack, lug wrench, and C-clampJack the car, remove the wheel, compress the caliper and replace the brake pads.BrakesHardActive collection
Remove, replace, and torque brake pads and rotors using a socket set and a torque wrench.Automotive repair shop with vehicle lifts, tool chests, and dedicated service bays.BrakesHardActive collection
Vehicle diagnosticsConnect the scan tool, read fault codes, inspect components and identify the fault.DiagnosticsHardActive collection
Automotive Service & MaintenanceDiagnosing and servicing vehicles including fluid, brake and mechanical work.GeneralHardActive collection

Episode fields

What fields does a maintenance episode carry?

Episode fields as documented on the site, and what each is useful for in maintenance work.
FieldAs documentedUse for maintenance tasks
Hand joints21 joints per hand, both hands, with a per-joint visibility flagWhich hand holds the part and which turns the fastener
contact_eventsThe moment the hand meets the itemWhen a hand takes hold of a filter, plug or terminal, and when it lets go
Action segmentsStart, end, verb and noun; 97 verbs, 512 nouns, human-reviewedOne labeled span per drain, swap, refill or tighten step
Instance masksStored at the depth resolution, 848×480; per-frame where the spec calls for themWhich part is which across frames
Timestamps and calibrationint64 nanoseconds per stream; intrinsics and extrinsicsAligning the head and wrist views
Fluid type, part type, torque value, success label, force or tactileNot statedNot stated

Delivery formats

Which formats are delivered, and what does each format’s own documentation say?

The delivery formats the site lists. The third column is taken only from the official pages linked under Sources.
FormatDelivered by FirsthandMaintainer documentation
RLDSListedEpisodes, each containing a dataset of steps
LeRobotListedSynchronized MP4 videos (or images) plus Parquet files
WebDatasetListedTar files; files of one sample share a basename
HDF5, zarr and Rerun .rrdListedNot described on this page

Not stated

What does the site not state about maintenance collection?

Details a buyer may ask about and where the answer lives.
TopicOn this siteWhere it is settled
Wiper, bulb or multi-point inspection rowsNot listed in the catalogWritten skill spec
Vehicle makes and types, service intervalsNot statedScoped in the spec
Fluid types, part types and torque valuesNot statedScoped in the spec
Number of vehicles, episodes, contributors or sitesNot statedQuote
How lifts are used in a taskOnly the setting line as listedWritten skill spec
Capture at a buyer’s own garage or facilityNot statedWritten skill spec
Torque, grip force or tactile dataNot stated for the standard captureNot offered as a stated stream
Success or quality labelsNot statedScoped in the spec
Which timeline applies to which rowNot statedQuote
Robot results or teleoperation dataNot stated; Firsthand captures people, not robotsYour own retargeting step

FAQ

Vehicle maintenance robot training data, answered.

Which vehicle maintenance tasks does Firsthand list?
Oil and fluid changes, filter replacement, battery testing and replacement, brake pad and rotor service, vehicle diagnostics and general service: thirteen rows in all, in one auto service setting.
Is there an oil change task in the catalog?
Yes. "Oil & Fluid Change" is described as "Drain engine oil, replace the filter, refill oil and top up coolant and other fluids", is listed as medium skill and is ready to deploy. Fluid and filter types are not stated.
Does the catalog include wipers, bulbs or a multi-point inspection?
No. No row names replacing wipers or bulbs, and none names a multi-point inspection. They would be scoped in a written spec.
What streams and labels come with a maintenance episode?
Ten hardware-synchronized streams within the 2 ms ceiling, 3D hand pose with visibility flags, contact events and action segments. No fluid-type, part-type, torque or success label is stated.
Do you capture force or tactile data for maintenance work?
The standard capture lists no force or tactile stream, and contact is described only by contact events. A separate glove tier has its own page, and this page makes no claim about it.
How are bystanders, customers and license plates protected?
Every participant signs a written release, faces and license plates are irreversibly blurred, VIN plates are blurred at ingest without exception, and a reviewer confirms no un-consented person is identifiable. The site says nothing separate about customers.
How fast can a custom vehicle maintenance dataset be collected?
From a signed spec to a first validated batch, a median of 48 hours where coverage is deep and three to four weeks where it is thin. The site does not say which applies to which maintenance row, so ask when you request the quote.
Do you capture robot or teleoperation data?
No. Firsthand captures people, not robots, so using the episodes for a specific robot means your own retargeting step. The license is perpetual and buyer-owned, and the executed order form governs.

Sources

Where are the delivery formats on this page documented?

  • RLDS (GitHub) (opens in a new tab)RLDS is an ecosystem of tools to store, retrieve and manipulate episodic data; the dataset is retrieved as a tf.data.Dataset of episodes where each episode contains a tf.data.Dataset of steps. github.com
  • LeRobot (GitHub) (opens in a new tab)The LeRobotDataset format is described as synchronized MP4 videos (or images) for vision and Parquet files for state and action data. github.com
  • WebDataset (GitHub) (opens in a new tab)WebDataset files are tar files in which files that make up a training sample share the same basename. github.com

Request a custom vehicle maintenance dataset.

Tell us the maintenance task, the order of steps and what counts as a reject. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.