Stereo Drywall Installation and Finishing Data

It is first-person video of a person hanging drywall panels and finishing the seams, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the two drywall rows in the site’s catalog: lifting and screwing panels to studs, and taping and mudding the seams. Stereo describes the capture, and MCAP describes the container. They are separate choices, and this page covers both.

The site does not state that stereo is already running on the drywall rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup, and a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

People reach this topic through several phrasings: drywall robot training data, stereo drywall hanging and taping video, an egocentric drywall mudding dataset, or custom construction finishing data collection. They describe one request from different angles, and we run it as custom collection, fast and on demand, against a spec you write. One phrasing needs a caution. The catalog states hanging, taping and mudding for drywall, but it states no sanding of drywall, so this page does not claim a drywall sanding dataset.

The egocentric drywall and wall finishing page covers the same two rows as capture data: the episode contents, the labels, the settings and privacy, with nothing on stereo depth or the MCAP container. The stereo layout marking and taping page counts the two drywall taping rows inside an eleven-row set that also holds measuring, marking and masking. The stereo surface finishing page covers sanding, drywall and painting rows together. This page does not repeat them.

It covers the drywall rows on their own, from hanging the panel through taping and mudding the seams, with the stereo and MCAP layer on top: what a calibrated pair adds to a panel, a seam and a corner, what MCAP is according to its own documentation, which streams can ship in the file, and what the site does not state. The rows are the same and the claims are as limited. This page states no board, compound, finish-level, tolerance, force or safety figure, because the site states none.

Two rows count, and both sit in the Active Construction Site environment of the Construction & Trades sector. "Drywall Hanging, Taping & Mudding" is described as "Lift and screw drywall panels to studs, tape joints and apply joint compound." It is rated Medium and its status is Ready to deploy. The second row, "Hang and tape drywall panels using a screw gun and taping knife," is described as "Hang drywall panels with a screw gun and tape and mud the seams with a taping knife." It is also rated Medium, and its status is Active collection. Both list the capture setup "Phone (head-mounted) / multiview rig". The first table below reproduces them as the catalog states them.

The site defines the statuses plainly. Ready to deploy means contributors and site access are set up and capture starts once a spec is signed. Active collection means recording is running now, so a spec can join an existing capture stream. The environment holds 15 rows in all: six are rated Hard, seven Medium and two Easy, and eleven are Active collection and four are Ready to deploy. Drywall is two of them. The site states no panel size, no wall or ceiling position, no room type and no task duration for either row.

Neighbouring rows are not counted. Painting & Priming masks edges and paints walls and has its own stereo page, and the insulation row fits insulation between studs and joists. The catalog’s sanding rows sit in the carpentry workshop and trades workshop bench environments and describe sanding panels and surfaces, not walls.

The two rows name their steps in their own words: lift and screw panels to studs, tape joints, apply joint compound, and work with a screw gun and a taping knife. The third table below sets each step beside what a stereo capture records and what is left to the spec. The pattern is the same for each: hand pose on the tool or the item, and the geometry of the panel, the stud line or the seam in the same frame pair.

Hanging is the step where depth describes a large flat panel meeting a frame. The rows say panels are lifted and screwed to studs. They do not say how big a panel is, whether the work is a wall or a ceiling, how the panel is held or how many people are involved. Taping and mudding are the steps where depth describes a seam: the joint being covered and the knife travelling over it. Whether a seam is resolvable in depth is Not stated.

Drywall work is about the state of a surface: whether a panel sits against the studs, whether a joint is taped, whether compound covers it. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the form of the panel and the seam sits in the same frame as the hands. The site gives no depth accuracy figure and no finish-quality label, so how well a panel edge, a screw head or a taped seam resolves in depth is Not stated.

The hard-environments page states that stereo keeps working outdoors where active depth sensors struggle, and that many programs capture both and sync them on one clock. Elsewhere the site notes that plain panels give a single camera little texture, and that a calibrated pair measures disparity between two views taken at a fixed, known baseline. Whether that carries over to a painted or taped wall is Not stated. Whether a drywall setting is a room, a shell or a site exterior is not something the rows state, so that is agreed in the spec.

The site describes two stereo setups and does not say which one a drywall order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

The site also defines two capture modes. A head-mounted phone records the first-person view of real work on site. A multiview rig with handheld grippers records a staged working set. Both drywall rows list "Phone (head-mounted) / multiview rig" and do not say which applies, so whether a batch is real work at a job site or a staged set is agreed in the spec. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. The second table below sets what is stated beside what is not.

The rows name two hand tools, a screw gun and a taping knife, and the catalog’s wording is all the site says about tools. A hawk, a mud pan, corner bead, a sanding pole and a lifting aid are not named, so this page names none of them and treats each as Not stated. Which tools are in frame is agreed in the spec.

Drywall keeps both hands busy: one holds or positions, the other drives or works the joint. 3D hand pose is delivered for both hands as 21 joints per hand with a per-joint visibility flag, and contact events mark when a hand meets the item. A spec can also ask for wrist views, so a grip the head camera loses is still recorded. Action segments divide the timeline into spans with start_ns, end_ns, verb and noun, so lifting the panel, taping the joint and applying compound can each be one labeled step.

Force and tactile readings are not in standard capture. The site separately offers a tactile glove tier as a pilot-ready option, with glove signals time-synced to a head-mounted stereo camera, and does not state that it has been used on drywall, so agree the fit first. The only language field the site ties to Firsthand capture is one task instruction per episode.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For drywall work the site states hand pose, a stereo-derived depth stream, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fifth table below sets each stream beside what the site states about it.

Hands are the first risk. A screw gun or a knife sits against the surface and the fingers wrap it, so the contact point is often hidden. The visibility flag on each joint is the answer the site gives, and contact events mark when a hand meets the item. A panel held up to the studs can also hide the joint being fixed, and a spec can ask for wrist views to cover that.

Depth is the second. The site says depth degrades on wet and reflective surfaces and masks low-confidence regions with per-frame confidence maps, and an episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. Wet joint compound is a wet surface, but the site does not say how depth behaves on it, so this page promises no figure.

Calibration is the third, because stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate, and an episode is rejected when sync breaches the ceiling for more than three consecutive frames. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it or an undocumented bystander appears. The site states no threshold for glare, motion blur or dust, so those are Not stated.

Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. A human reviewer checks each episode against the task spec. For a drywall task, the spec decides what the manipulated object is: the panel, the tool or the joint.

Every contributor signs a written release before capture, and each delivered episode links to that record. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and a human reviewer confirms that no un-consented person is identifiable in any frame. A construction setting can have other people nearby, so settle bystander handling in the spec. The site states no contributor safety protocol, no dust or building-code claim, no PPE guidance and no safety certification for drywall, and this page describes none.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim off-the-shelf coverage of drywall. Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. One drywall row is Ready to deploy and the other is Active collection, but the site does not state how deep coverage is, and the construction guide treats construction and skilled trades as a thinner-coverage setting, so plan on a quote rather than assuming the faster figure. A pilot reaches its first delivery in two to three weeks and a standing program reaches full rate in four to six weeks, assuming capture coverage is in place. Collections are billed per validated hour, and capture at your own facility is listed as not offered. After the first batch, on-demand top-ups run against the same accept criteria.

Describe the drywall steps and their order, what counts as a completed step, the surface and the setting, and the variation your pipeline is missing, such as seams, corners or panel edges. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The sixth table below shows what each spec element decides.

Where this site does not state something, this page says Not stated rather than filling the gap. There is no listed label for board size or type, compound product, drying time, finish level, joint smoothness, tape flatness, tolerance or placement accuracy, no sanding of drywall, no corner bead or patching task, and no tool specification beyond the screw gun and taping knife the rows name. There is no safety, PPE, dust or building-code guidance, and nothing here claims a training outcome or a model improvement.

The tradeoff to plan for is the embodiment gap between a human hand and a robot platform working a wall, which retargeting or pairing with robot data has to close. What you get is human first-person demonstrations in which the hands, the panel, the seam and the step structure are explicit. Counts of contributors, hours and tasks are set in your skill spec.