Stereo Produce Sorting and Packing Data

It is first-person video of a person doing produce sorting and packing work: inspecting, cleaning and sorting fresh strawberries with sorting trays and collection bins, weighing, packing, sealing and labeling them in clamshell containers with a bench scale and a label gun, and grading produce by size and quality and packing it into crates. It is recorded from their own point of view with two calibrated cameras, with hand pose and sensor streams alongside. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as the capture setup for the strawberry rows and a multiview rig with handheld grippers for the greenhouse rows, so a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. The site does not state grading standards, defect labels, weight readings or decoded label text, so this page promises none. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Five, all in the Agriculture & Grounds sector, all rated Easy, and all listed as Available on request. Three are in the Produce Packing Facility environment: "Inspect, clean, and sort fresh strawberries, removing stems and damaged fruit, using sorting trays and collection bins.", "Sort, weigh, pack, and label fresh strawberries into clamshell containers using a bench scale and a label gun." and "Weigh, pack, seal, and label fresh strawberries into clamshell containers using a bench scale and a label gun." Two are in the Greenhouse, Nursery & Garden Centre environment: "Pack produce crate", described as "Pack graded produce into crates.", and "Sort + grade produce", described as "Grade produce by size and quality."

None of the five is listed as Active collection or Ready to deploy, and the site says nothing about delivered hours or delivered stereo footage for them, so this page claims none. It names only the tools and items that appear in the rows: sorting trays, collection bins, a bench scale, a label gun, clamshell containers and crates. The first table below reproduces the five rows as the catalog states them. The Farm & Field Operations row "Crop Sorting & Grading" is listed as Ready to deploy but is a different environment, so it is not relied on here.

The egocentric produce packing and grading page covers six produce rows with egocentric catalog treatment and has no stereo or MCAP treatment. The stereo order packing and taping page gives stereo and MCAP treatment to four warehouse rows, and the stereo warehouse sorting and labeling page does the same for three sort-and-label rows. Neither warehouse page names fruit or produce.

This page is the stereo and MCAP treatment of the five produce sorting, grading and packing rows, with its own stream map and failure modes. The comparison table below sets the pages side by side, with each row status stated as the live catalog lists it.

Produce work is about where an item is, which tray or container it goes to and what the hand does to place it: a berry lifted from a tray, a clamshell filled on a packing line, a crate packed with graded produce. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the item, the container and the hands share a frame. The site gives no depth accuracy figure for these rows and no grade, defect or placement label, so what a buyer does with that depth is their pipeline question.

A stereo pair adds depth, not a verdict on the fruit. The site states 1080p per eye for dataset 06 and gives no figure for how well small items such as strawberries resolve, and it does not state that fruit quality is judged from the video, so these are Not stated and a spec should check them on a Spec Pilot. Capture at your own facility is listed as not offered.

The site states 3D hand pose of 21 joints per hand with a per-joint visibility flag, and contact and grasp events that mark when fingers meet a tool or an item. In these rows that is the hands picking and turning fruit at a sorting tray, setting it on a bench scale, placing it in a clamshell, sealing the container and applying a label with a label gun, or packing graded produce into a crate.

The site does not say whether the fruit, the clamshell, the bench scale or the label gun counts as the object for contact events, so settle it in the spec. Wrist cameras give a closer view of the hands, and a multiview rig adds synchronized third-person angles when the spec asks for them.

The site describes two stereo setups and does not say which one a produce order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a produce order uses is Not stated on the site, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. Bench layout, lighting, scale and label-gun placement and camera clearance are Not stated, so any requirement is settled in the spec.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For these rows the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Hand occlusion of the item is the first risk. Strawberries are small, and fingers, a tray edge or a clamshell lid can hide the fruit being held. The visibility flag on each joint is the answer the site gives, contact events mark when a hand takes hold of an item, and a spec can ask for wrist views or a second angle so a grip the head camera loses is still recorded. The site notes that reach and grasp frames are where blur and occlusion show up. An episode is rejected when the manipulated object is out of frame for more than 15 percent of it.

Glossy or wet surfaces are the second. The first row description names washing stations, and the site states that depth degrades on wet and reflective surfaces and ships per-frame confidence maps so a buyer can mask rather than guess. How a stereo pair behaves on wet fruit, a glossy lid or a steel bench is Not stated. An episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid, and depth is flagged rather than interpolated where it is unreliable.

Fast repetitive hand motion is the third: sorting and packing repeat the same pick, check and place many times in a row. The head camera is global-shutter, which avoids rolling-shutter skew, but the site gives no figure for blur at packing speed, so how a stereo pair behaves at that pace is Not stated. Scale and label legibility is the fourth. The site gives no legibility figure and does not state that a scale reading or label text is read or transcribed, so this page promises none. Small soft fruit is the fifth: the site states no handling specific to deformable items, so how a stereo pair represents a berry that is pressed or turned is Not stated and should be checked on a Spec Pilot.

Sequence and calibration close the list. The rows chain sorting, weighing, packing, sealing and labeling, so a spec naming each step and where it starts and ends gives a sequence that can be sliced by step, while a spec saying "pack strawberries" gives one long action segment. Stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when an undocumented bystander appears, and a reject log ships with every batch. Sort or pack rates, throughput, defect or grading criteria, farm, grower, retailer or customer names, scale or label-gun brands and food-safety certifications are Not stated and are not claimed here.

For every delivered episode the site states that faces, plates and screens are blurred irreversibly, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data, and an undocumented bystander is a stated reject criterion.

Text printed on labels is not listed as a blur category on the site, so this page does not claim it is blurred. The site does not state a rule specific to a packhouse or a greenhouse, so this page makes no such claim. Settle it in the spec.

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. The site lists no off-the-shelf produce sorting or packing dataset. Stereo produce sorting and packing in MCAP is therefore custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. The off-the-shelf stereo and hand-tracking video page shows what the rig delivers, and the custom order applies it to your task.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. All five rows are Available on request, and the site does not say which speed applies to a stereo produce order, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Describe the steps and where each starts and ends: inspecting and sorting, weighing, packing into clamshell containers or crates, sealing and labeling, or grading by size and quality. Name the tools you expect to see, whether an episode covers one container, one crate or a longer stretch of work, what counts as a reject and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.