Stereo Filling Line Operation Data

It is first-person video of a person running a filling line or a production line: starting the line, reading gauges, adjusting settings on the panel and clearing minor jams, or feeding product onto the line, monitoring the machines and packing finished goods at the end. It is recorded from their own point of view with two calibrated cameras, with hand pose and sensor streams alongside. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone with a multiview rig as their capture setup, so a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. The site does not state machine or line brands, product names, fill volumes, line speeds, cycle times, throughput, or plant, brand or customer names, so this page promises none. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Two, both in the Food & Beverage Manufacturing Plant environment of the Food & Beverage sector. "Operate and monitor a filling line using control panels and gauges" is described as "Start the filling line, read gauges, adjust settings on the panel and clear minor jams." "Production Line Operation" is described as "Feed product onto the line, monitor the machines and pack finished goods at the end." Both are rated Medium.

Both are listed as Ready to deploy, which the catalog defines as contributors and site access already set up. The site says nothing about delivered hours or delivered stereo footage for either row, so this page claims none. The only tools the rows name are control panels and gauges. The first table below reproduces the rows as the catalog states them.

The egocentric filling line operation page covers the same two rows as first-person catalog capture, with no stereo or MCAP treatment. The stereo food plant sanitation washdown page covers the two wet cleaning rows of the same plant. The stereo injection molding machine tending page covers two press-tending rows in another environment, and the stereo heavy machinery operation page covers one lever-and-panel row rated Hard.

This page is the stereo and MCAP treatment of the two line-operation rows, with its own stream map and failure modes. The second table below sets the pages side by side.

Line operation switches attention between a panel and the product. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the hands, the panel and the line share a frame. The site gives no depth accuracy figure for these rows and no label for a gauge value, a panel state or a jam, so what a buyer does with that depth is their pipeline question.

A stereo pair adds depth, not a reading of the display. The site states 1080p per eye for dataset 06, and whether that suffices to read a gauge face is Not stated, so a spec should check it on a Spec Pilot. The site also states that Firsthand does not capture at a customer’s own facility.

The site states 3D hand pose of 21 joints per hand with a per-joint visibility flag, and contact and grasp events that mark when fingers meet a tool or an item. For a panel, that is the label that carries how the hands press, turn and hold while a setting is adjusted or a jam is cleared.

The site does not say whether a panel control or a gauge counts as the item for contact events, so settle that in the spec. Neither row names gloves, and how hand pose behaves on a gloved hand is Not stated. Wrist cameras give a closer view of the hands, and a multiview rig adds synchronized third-person angles when the spec asks for them.

The site describes two stereo setups and does not say which one a filling line order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams an order uses is Not stated on the site, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. Panel layout, lighting and camera clearance are Not stated, so any requirement is settled in the spec.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For these rows the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Hand occlusion at the panel is the first risk. Fingers disappear behind controls, which is why each hand joint carries a visibility flag, and a spec can ask for wrist views or a second angle. An episode is rejected when the manipulated object is out of frame for more than 15 percent of it. For a line task the spec decides what that object is: the product, a panel control or both.

Gauges, panel displays and glare are the second. The site describes glossy or dark surfaces as places where a single camera loses information, and it states that depth degrades on wet and reflective surfaces and ships per-frame confidence maps so a buyer can mask rather than guess. How depth behaves on a gauge face or a panel display in particular is Not stated. Automated detection also finds screens at ingest and blurs them irreversibly, and how that applies to a panel display or a gauge is Not stated, so if your task needs a readable display, raise it when you scope the collection rather than assuming it will be legible.

Moving machinery and close working distance are the third. The site describes the camera as sitting on the head, close to what a robot’s onboard camera would see at a line, but gives no depth figure at that distance, and it states nothing about how moving machinery in frame affects a stereo pair. An episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid.

Calibration is measured at the start and end of every session under a 0.28 px RMS gate, an episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, for an undocumented bystander or for a mismatch against the named skill spec, and a reject log ships with every batch. Machine and line brands, fill volumes, line speeds, cycle times, throughput, safety rules and food-safety certifications are Not stated and are not claimed here.

For every delivered episode the site states that faces, plates and screens are blurred irreversibly, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data, and an undocumented bystander is a stated reject criterion.

The site does not state a rule specific to a plant floor, brand labels on product or branded packaging, so this page makes no such claim. Settle it in the spec.

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. No listed dataset names filling line or production line operation. S5, a stereo dataset of about 2,000 hours with a head stereo rig and two wrist cameras, lists "Manufacturing across 50+ factories: appliance and LED assembly, converter lines, packaging, fastening, tool use", and the site does not state that it contains either row.

Stereo filling line operation in MCAP is therefore custom collection, not an off-the-shelf purchase: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. Both rows are Ready to deploy, and the site does not say which speed applies to this order, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Describe the steps and where each starts and ends: starting the line, reading the gauges, adjusting a setting, clearing a jam, feeding product, monitoring the machines. Say whether an episode covers a start-up, a monitored run or one intervention, since the site does not set an episode length. Name what counts as a reject and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.