Stereo HVAC Installation and Service Data

It is first-person video of a person doing HVAC work, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the ones the site’s catalog lists: positioning and mounting units, connecting ducting, refrigerant lines and wiring, replacing and tuning filters, mounting a thermostat and connecting its control wires, and inspecting a unit on a service call. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup, and a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

The egocentric HVAC service page covers the five rows as a task family: what each row says, the environments, what an episode contains, privacy, speed and licence. This page does not repeat it. It takes the same five rows and covers the two choices layered on them: a calibrated stereo pair for depth, and MCAP as the delivery container.

The stereo plumbing and stereo electrical pages cover their own rows in the same Trade Service Calls environment, and neither counts the HVAC rows. Here the HVAC rows stand alone, because a trade-service or home-robotics lab asking about units, filters and thermostats wants those five rows specified together. No equipment type, rating, refrigerant or code appears on this page beyond the catalog’s own words, because the site states none.

Five rows count. In Trade Service Calls: “HVAC Equipment Installation”, described as positioning and mounting HVAC units, connecting ducting, refrigerant lines and wiring, then commissioning; “Filter Swap & System Tune-Up”, described as replacing filters, cleaning coils, checking pressures and tuning the system; “Replace and tune HVAC filters using screwdrivers and gauge sets”, described as opening the unit, swapping the filters, connecting gauges and adjusting settings to spec; and “Thermostat & Control Wiring”, described as mounting the thermostat, connecting control wires to terminals and testing the heating and cooling calls. In On-Site Home Repair: “HVAC service call”, described as inspecting the unit, replacing filters or parts, checking refrigerant and testing the system. The first table below reproduces each as the catalog states it.

Two rows are rated Medium and three Hard, and all five list the same capture setup. The four Trade Service Calls rows are Ready to deploy, which means contributors and site access are set up and capture starts once a spec is signed. The service call row is Available on request, which means it is scoped and recruited to spec when a buyer asks. The site names no building, contractor, equipment brand or tool model for any row, so this page names none.

HVAC work is about placement and connection: where a unit is mounted, where a filter sits when it comes out, where a gauge is attached, which terminal a control wire lands on. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the shape of the work sits in the same frame as the hands. The site gives no depth accuracy figure for these rows and no placement or connection label, so what a buyer does with that depth is their pipeline’s question.

The site’s stereo material also says that in a glare-heavy or low-texture scene a single-frame depth guess gets worse, and that a calibrated pair measures disparity between two views at a fixed, known baseline instead. It adds that stereo keeps working outdoors where active depth sensors struggle. How a metal casing or a filter face behaves under that description is not stated, and nor is whether a job is indoors, outdoors or in a loft or plant room.

The rows name the things under the hands: units, ducting, refrigerant lines, wiring, filters, coils, gauges, the thermostat and control wires going to terminals. Depth from a stereo pair puts those in three dimensions, so how a unit is positioned, how a filter is drawn out and a new one seated, and how a control wire approaches a terminal are in the capture. Whether the unit is wall-mounted, floor-standing, on a roof or indoors is not stated.

The site lists no label for any of them: no unit type, size or position, no filter size, no pressure reading, no wire colour or terminal designation, no commissioning result. It states that metric depth on sub-centimetre parts is flagged rather than interpolated, and does not say whether a terminal screw or a control wire counts as such a part. If a policy depends on any of those, say so at the start so the capture is planned around it.

The site states 3D hand pose of 21 joints per hand with a per-joint visibility flag, and contact and grasp events that mark when fingers meet a tool or a part. In HVAC work that is the grip on a screwdriver while a unit is opened, the hold on a gauge set as it is connected, the hand that draws out a filter, and the fingers that place a control wire at a terminal.

These are the moments a head camera loses most often, because the tool or the unit sits between the lens and the fingers. A spec can ask for a wrist view so a grip the head camera loses is still recorded, and the visibility flag marks the joints that were hidden. The only tools the rows name are screwdrivers and gauge sets. The site states no force, torque or tool-pose stream, and does not say whether a filter, a gauge or a wire counts as the object in a contact event, so settle that in the spec.

The rows describe four short sequences. Position and mount a unit, connect ducting, refrigerant lines and wiring, then commission it. Replace filters, clean coils, check pressures and tune the system. Open the unit, swap the filters, connect gauges and adjust settings to spec. Mount a thermostat, connect control wires to terminals and test the heating and cooling calls. The service call row adds an inspection first, then replacing filters or parts, checking refrigerant and testing the system. The order is the order the catalog gives.

A decision comes before capture: what counts as one attempt. It could be one filter swap, one thermostat, one unit, or one whole service call. The site does not set that length, so the skill spec does, along with which verbs and nouns the segments use. The site also states one task instruction per episode, which is the only language data it describes for an episode.

The site describes two stereo setups and does not say which one an HVAC order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

The catalog lists a head-mounted phone or multiview rig for all five rows. Which rig, resolution and streams an HVAC order uses is Not stated, and the second table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. Capture at your own facility is listed as not offered.

Glare and low texture are the first risk. The site notes that glare-heavy and low-texture scenes make single-frame depth worse, and that depth degrades on wet and reflective surfaces while confidence maps mask the low-confidence regions. An episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. The site states a low-light share only for its automotive dataset, so lighting handling for HVAC rows is Not stated, and so is the handling of dust or tight, enclosed spaces.

Occlusion and motion blur are the second. The head camera is global-shutter, which avoids rolling-shutter skew, and the site notes that reach and grasp frames are where blur and occlusion show up. An opened unit puts a casing, a hand and a tool close together, so hand pose carries a visibility flag for joints they hide. Calibration is the third, because stereo depth is only as good as the rectification behind it: it is measured at the start and end of every session under a 0.28 px RMS gate.

Framing and bystanders are the fourth. An episode is rejected when the manipulated object is out of frame for more than 15 percent of it, or an undocumented bystander appears. The third table below sets each risk beside the site’s handling. The site does not describe how a unit that fills the view is handled.

Every batch is validated against signed accept criteria before it ships, a reject log ships with it, and rejected items are recaptured at our cost. Delivery is rolling, per validated batch, and billing is per validated hour. Each episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. Footage with bystanders who cannot be de-identified is discarded, which matters on a service call in a client’s home. Access to a client’s home or site is a condition the spec sets. The site makes no refrigerant-handling, safety, protective equipment, licensing, certification, code or compliance claim for any HVAC row, and neither does this page.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression, with optional chunking and CRC checksums.

Native reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For HVAC the site states a stereo pair, a stereo-derived depth stream with confidence maps, hand pose, IMU, contact events, action segments and calibration elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

The site states delivery in RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd, and MCAP can ship alone or beside any of them. HVAC episodes follow the same episode schema as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes. In the HDF5 layout the site documents, hand pose sits at annotations/hand_joints and action segments at annotations/action_segments.

Which container suits a lab depends on its reader and its training loop, and the site does not rank them. The compare page on MCAP and HDF5 sets the two side by side, and the delivery formats guide covers the rest. Say in the spec whether MCAP is the only delivery or sits beside a training format.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim off-the-shelf coverage of the HVAC rows. A lab can use the stock stereo footage to check the rig and the format first, then commission the HVAC tasks it needs. Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. Four of the five rows are Ready to deploy, and the service call row is recruited to your spec first, so ask which applies to each row when you request it. Collections are billed per validated hour. The supply page covers how a recurring flow is set up.

Describe the sequence your policy needs: mount a unit and connect it; open a unit, swap filters and connect gauges; mount a thermostat and wire its controls; or work a full service call. State what counts as a completed step, the setting, and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. Equipment types, ratings and tool models are agreed in the spec because the site does not state them. The fifth table below shows what each spec element decides.