Stereo Welding and Metal Fabrication Data

It is first-person video of a person working metal, recorded from their own point of view with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the ones the site’s catalog lists for welding, metal fabrication, layout, pipe cutting and fitting, grinding and deburring, and weld inspection. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup, and a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

The egocentric welding and metal fabrication page covers the tasks: the catalog rows, the episode schema, the delivery formats and what that page does not state. This page does not repeat it. It covers the two choices layered on top of the same rows: a calibrated stereo pair for depth, and MCAP as the delivery container.

The stereo heavy machinery operation page covers cab and control work on equipment, and the stereo surface finishing page covers sanding, drywall and painting. Welding and fabrication share some ground with both, in shop machinery and in grinding, but the rows, the hand work and the open questions are different. The rows are the same as on the egocentric page and the claims are as limited. This page states no amperage, filler, shade, bead-quality or certification figure, because the site states none.

Twelve rows count, in five families. Two broad rows sit in the Heavy Industry & Fabrication environment: Metal fabrication and Welding. Ten sit in the Metal Fabrication & Welding Workshop environment: two further welding and fabrication rows, two layout rows, two pipe rows, two grinding rows and one weld inspection row. The first table below reproduces each as the catalog states it.

Statuses are Available on request for the two heavy industry rows and Ready to deploy for the ten workshop rows, which means contributors and site access are set up and capture starts once a spec is signed. The knife-forging row in the same workshop environment is covered on the knifemaking page and is not counted here. Skill ratings run from Medium to Hard, and none is rated Easy. The site names no shop, brand, worker, material or machine model for any row.

Fabrication is about the state of a part: whether a pipe end is bevelled, whether a frame is square before the tack, whether a seam is ground flush. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the form of the part and the fixture sits in the same frame as the hands. The site gives no depth accuracy figure for these rows and no weld-quality label, so what a buyer does with that depth is their pipeline’s question.

The hard-environments page states that stereo keeps working outdoors where active depth sensors struggle, and that many programs capture both and sync them on one clock. Whether a fabrication setting is a workshop bay, a heavy-industry floor or a yard is not something the catalog rows state beyond the environment names, so that is agreed in the spec.

The site states 3D hand pose of 21 joints per hand with a per-joint visibility flag, and contact and grasp events that mark when fingers meet a tool or a part. In fabrication that is the grip on a clamp, the hold on a torch or machine, the two-handed guide of an angle grinder and the fingers steadying a workpiece on a bench.

These are the moments a head camera loses most often, because the tool, a guard or the workpiece sits between the lens and the fingers. A spec can ask for a wrist view so a grip the head camera loses is still recorded, and the visibility flag marks the joints that were hidden. The site states no grip-force, tool-pose or torch-angle stream, and this page does not claim one.

The site describes two stereo setups and does not say which one a welding or fabrication order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

The egocentric welding page states that a head-mounted phone records the first-person view and that a multiview rig adds synchronized third-person angles when the spec asks for them. Which rig, resolution and streams a welding order uses is Not stated, and the second table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling.

Reflective surfaces are the first risk. The site says depth degrades on wet and reflective surfaces, and confidence maps mask the low-confidence regions. An episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. The site does not say how depth behaves on bare, polished or ground metal, so this page promises no figure.

The arc is the second, and the site is plain about it: it does not say how a recording handles the bright light of an arc, a helmet or a shield, and exposure handling is Not stated. Raise it in the spec before capture. Motion blur and occlusion are the third. The head camera is global-shutter, which avoids rolling-shutter skew, and the site notes that reach and grasp frames are where blur and occlusion show up. Hand pose carries a visibility flag for joints a clamp or a grinder hides.

Calibration is the fourth, because stereo depth is only as good as the rectification behind it: it is measured at the start and end of every session under a 0.28 px RMS gate. The fifth is framing: an episode is rejected when the manipulated object is out of frame for more than 15 percent of it, or an undocumented bystander appears. No weld-defect, bead-quality or completion label is listed on the site.

Every batch is validated against signed accept criteria before it ships, a reject log ships with it, and rejected items are recaptured at our cost. Delivery is rolling, per validated batch, and billing is per validated hour. Each episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The site does not state a separate rule for drawings, labels or machine displays in a shop, so this page does not claim they are blurred. The site makes no safety, protective equipment or compliance claim for any welding or fabrication environment, and neither does this page.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression, with optional chunking and CRC checksums.

Native reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For fabrication work the site states a stereo pair, a stereo-derived depth stream with confidence maps, hand pose, IMU, contact events, action segments and calibration elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The third table below sets each stream beside what the site states about it.

The site states delivery in RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd, and MCAP can ship alone or beside any of them. Welding and fabrication episodes follow the same episode schema as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes.

Which container suits a lab depends on its reader and its training loop, and the site does not rank them. The compare page on MCAP and HDF5 sets the two side by side, and the delivery formats guide covers the rest. Say in the spec whether MCAP is the only delivery or sits beside a training format.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim off-the-shelf coverage of the welding, layout, pipe or grinding rows. A lab can use the stock stereo footage to check the rig and the format first, then commission the fabrication tasks it needs. Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. Ten of the workshop rows are listed as Ready to deploy, but the site does not say which coverage level applies to a given row or setting, so ask when you request the quote. Collections are billed per validated hour, and capture at your own facility is listed as not offered.

Describe the parts, joints and sequence your policy needs: layout, cutting, fitting, tacking, welding, grinding, inspection. State what counts as a completed step, the setting, and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. Welding process, metal, thickness, arc handling and machine or torch models are agreed in the spec because the site does not state them. The fourth table below shows what each spec element decides.