Stereo Painting and Coating Data

It is first-person video of a person applying a material to a surface, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the ones the site’s catalog lists for painting, priming, cut-in and staining. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup, and a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

The egocentric painting and coating page covers the tasks: the catalog rows, the labels, the privacy steps and what that page does not state. The stereo surface finishing page covers nine rows across sanding, drywall and painting, and the stereo layout marking and taping page covers measuring, marking, taping and masking. This page does not repeat them.

It narrows to the painting and coating rows only, and asks what a painting-robotics or finishing-automation lab needs from a stereo capture of them: the surface geometry around an edge, the hand on the tool, the failure modes of a wet coat and the container the data arrives in. The rows are the same as on the sibling pages and the claims are as limited. This page states no paint type, coverage, film thickness or finish-quality figure, because the site states none.

Four rows count. In the active construction site environment the catalog lists "Painting & Priming", which masks edges, rolls primer and paint onto walls and cuts in corners with a brush, and "Prime and paint interior walls using rollers and brushes", which rolls primer then paint and brushes cut-in at edges and trim. The trades workshop bench lists "Roll + cut-in paint", which rolls a panel and cuts in edges. The carpentry and furniture workshop lists "Staining & Finishing", which applies stain, sealer or lacquer with rags, brushes or a spray gun and sands between coats. The first table below reproduces each as the catalog states it.

Statuses range from Ready to deploy and Active collection to Available on request, and the bench row is recorded with a multiview rig and handheld grippers instead of the phone or multiview rig used on the others. Canvas painting and mural work in the art studio are artistic rows and are covered on the egocentric painting page, not counted here. The site states no paint type, tool model, wall size or surface material for any row.

Painting is about the state of a surface: whether an edge is clean, whether a corner is covered, whether the roller has left the wall wet or dry. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the form of the wall, the corner and the trim sits in the same frame as the hands and the tool. The site gives no depth accuracy figure for these rows and no finish-quality label, so what a buyer does with that depth is their pipeline’s question.

A cut-in line is a good test of why geometry matters. The brush follows the meeting of two surfaces, a wall and a ceiling or a wall and trim, and the position of that edge in three dimensions is what a policy has to follow. A flat RGB frame shows the edge as a colour change. A stereo pair adds where the two planes actually sit. The site does not state a cut-in accuracy figure and promises none.

The site describes two stereo setups and does not say which one a painting order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a painting order uses is Not stated on the site, and the second table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, an episode is accepted only if RMS reprojection stays under 0.28 px, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling.

Hand pose is delivered for both hands, as 21 joints per hand, each with a visibility flag. On a painting task that is the label that carries how the hand sits on a brush handle, how the other hand steadies a roller frame or a tin, and how the grip changes between a roller pass and a cut-in stroke. The site’s tool use page says grip changes live in finger-level detail that RGB video alone cannot resolve, which is the reason hand pose sits beside the video.

Contact events mark the moment the hand meets an item. The site does not say whether a roller, a brush, a paint tin, a masked edge or the wall counts as the item, so the spec is the place to settle what a contact event means for your task. A spray gun is named in the wording of one catalog row, "Staining & Finishing", and nowhere else. The site states no spray pressure, nozzle, booth or industrial finishing line, so spray work is a spec and Spec Pilot question, not a stated capability.

Wet and reflective surfaces are the first risk. Fresh paint, primer, stain and lacquer leave wet, glossy surfaces, and the site says depth degrades on them. Confidence maps mask the low-confidence regions, and an episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. The site does not say how depth behaves on a particular fresh coat, so this page promises no figure. The statement that painted and oily surfaces absorb a depth projector pattern is made for under-vehicle capture only, and it is not extended to walls here.

Hands are the second. The tool sits against the surface and the fingers wrap it, so the contact point is often hidden. The visibility flag on each joint is the answer the site gives, and a spec can also ask for wrist views, so a grip the head camera loses is still recorded. Occlusion by the tool itself is the same problem seen from the other side: a roller and its frame can cover the very patch being coated. The third table below sets each failure beside what the site states about it.

Calibration is the third, because stereo depth is only as good as the rectification behind it. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it, when sync breaches the ceiling for more than three consecutive frames, when calibration drifts beyond tolerance, or when an undocumented bystander appears, and a reject log ships with every batch. The site does not state a motion blur threshold for fast brush or roller strokes, and no glare figure, so neither is promised here.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For painting work the site states hand pose, a stereo-derived depth stream, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. A human reviewer checks each episode against the task spec. For a painting task, the spec decides what the manipulated object is: the tool, the patch of wall or the panel being coated.

Every contributor signs a written release before capture, and each delivered episode links to that record. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and a human reviewer confirms that no un-consented person is identifiable in any frame. Construction sites and shared workshops can have other people nearby, so settle bystander handling in the spec for the setting you choose. The site states no contributor safety protocol and no safety certification for any painting row, and this page describes none.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim off-the-shelf coverage of the painting rows. Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The site does not say which applies to the painting rows, so ask when you request the quote. Collections are billed per validated hour, and capture at your own facility is listed as not offered. After the first batch, on-demand top-ups run against the same accept criteria, so you can add surfaces, tools or steps without rewriting the spec.

Describe the painting steps and their order, what counts as a completed pass, the surface and the setting, and the variation your pipeline is missing, such as cut-in along trim or masking an edge. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.

Where this site does not state something, this page says Not stated rather than filling the gap. There is no listed label for paint or finish type, coverage, coat count, wet-film or coating thickness, stroke direction, surface-prep state or finish quality, no force or tactile stream, and no paint, tool or surface specification. There is no spray booth or industrial finishing claim, no ventilation, PPE or safety guidance, and nothing here claims a training outcome or a model improvement.

The tradeoff to plan for is the embodiment gap between a human holding a brush or roller and a robot platform working a surface, which retargeting or pairing with robot data has to close. Counts of contributors, hours and tasks are set in your skill spec.