Stereo Construction Site Cleanup Data

It is first-person video of a person doing the cleanup work the catalog lists for an active construction site, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup, and a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

The egocentric construction site cleanup page covers the same two rows as RGB, depth and hand-pose task video, with its streams, privacy wording and what is not stated. The stereo construction crews page covers all 15 Active Construction Site rows, of which site cleanup is one. The stereo floor mopping page covers nine floor-care and janitorial rows, a different environment. This page does not repeat them.

It covers the two layers on top of the two cleanup rows: a calibrated stereo pair for depth, and MCAP as the container. The page states no debris type, weight, volume, waste-handling rule, standard or certification, because the site states none.

Two, both in the Active Construction Site environment under the Construction & Trades sector. "Construction Site Cleanup & Debris Removal" is described as clearing debris, sorting waste and preparing sites after works. "Sweep and haul construction debris using brooms and wheelbarrows" is described as sweeping up debris, loading it into wheelbarrows and hauling it to the site skip. Both are rated Easy, listed as Active collection and listed with a head-mounted phone or multiview rig. The first table below reproduces them as the catalog states them.

The tools and steps on this page are only those in the rows: brooms, wheelbarrows, a site skip, and clearing, sorting, sweeping, loading and hauling.

Cleanup is about the state of a place: what is on the floor, what has been moved, what is left, and the route between the work area and the skip. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the form of the debris and the floor is in the same frame as the hands and the tool. The site gives no depth accuracy figure, no clearance label and no outcome label such as site clear, so whether a cleared area is resolvable in depth is Not stated.

The hard-environments page states that stereo keeps working outdoors where active depth sensors struggle. Whether a cleanup site is indoors or outdoors is not stated on the rows, so it is agreed in the spec.

The site describes two stereo setups and does not say which one a cleanup order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a cleanup order uses is Not stated. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling.

Hands are the first risk. A broom handle sits in the fist, a wheelbarrow handle is gripped low and a load can block the view, so the contact point is often hidden. The visibility flag on each joint is the answer the site gives, contact events mark when a hand meets the item, and a spec can ask for wrist views so a grip the head camera loses is still recorded.

Depth is the second. The site says depth degrades on wet and reflective surfaces and masks low-confidence regions with per-frame confidence maps, and an episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. Calibration is the third, measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it or an undocumented bystander appears, and a reject log ships with every batch.

The site states no handling for dust, low light, glare or motion blur on these rows, so this page promises none. The second table below sets each failure mode beside what the site says.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For cleanup work the site states hand pose, a stereo-derived depth stream, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The third table below sets each stream beside what the site states about it.

Off-the-shelf covers what is already in stock. The industrial and skilled-trade category lists datasets 02, 03 and 04, covering work such as machinery, auto repair, carpentry, construction, factory floors and inspection, and the stereo category lists the stereo head rig. Neither listing names site cleanup or debris removal, so this page claims no off-the-shelf coverage of the two rows.

Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The site states no provider count, cadence, hours or volume beyond that wording, so they are quoted against your spec. The fourth table below sets the two sources side by side.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours. Construction and skilled trades is described as a thinner-coverage setting, where a first batch typically takes three to four weeks, because vetting and scheduling operators with site access is the bottleneck, and we quote that up front. The site does not say which applies to these two rows, so ask when you request the quote.

A Spec Pilot, a small and fast first batch, proves the spec and accept criteria on real footage before you scale. Billing is per validated hour, and rejected items are recaptured at our cost.

A shared jobsite can put other trades and crew in frame. For every delivered episode the site states that faces and screens are detected and irreversibly blurred at ingest, a reviewer confirms the result, and footage that cannot be de-identified is discarded. Every episode links to a signed contributor release through a documented consent chain. The site states that Firsthand does not capture at a customer facility, and it states no rule for specific sites or contractors, so the site is agreed in the spec.

The buyer owns the dataset under a perpetual, irrevocable, non-exclusive-by-default licence, with exclusivity available. This page makes no safety, standards or certification claim for cleanup work.

Describe the tools, the debris in your own terms, the area and the haul route, the steps in order, what counts as one completed pass, and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, which confirms the spec on real footage.