Stereo Tractor and Farm Equipment Operation Data

It is first-person video of a person operating farm equipment, recorded with two calibrated cameras, with hand pose and sensor streams alongside. It is also called tractor operation robot training data or an egocentric agricultural equipment dataset. Stereo describes the capture, and MCAP describes the container.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig, so a stereo pair is agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Two, listed in lib/commercial-catalog-data.ts, both in Farm & Field Operations under the Agriculture & Grounds sector, and both rated Hard. "Farm equipment operation" reads "Operate tractors, mowers or loaders and attach and detach implements" and is Active collection. "Tractor & Implement Operation" reads "Drive the tractor, hitch implements and operate them across the field" and is Ready to deploy.

The site defines Active collection as recording running now and Ready to deploy as contributors and site access already set up. Those describe the catalog rows, not delivery dates. The site states no delivered hours and no delivered stereo footage for either row, so this page claims none.

This page is the stereo and MCAP treatment of the two equipment rows in Farm & Field Operations. The egocentric farm and field task page covers all 21 rows of the environment as first-person video, including both of these, and stereo is not its subject. The egocentric heavy machinery page covers one different row, working the levers and control panels of presses, cranes or machinery from the operator station, in Heavy Industry & Fabrication. The stereo heavy machinery page adds stereo and MCAP to that same single row.

The stereo crop harvesting page covers four rows about picking and cutting produce, which is plant work rather than equipment work. No stereo forklift page exists on the site, so none is compared. The second table below sets the pages side by side.

The catalog lists "Phone (head-mounted) / multiview rig" for both rows. The site does not state a stereo pair for either of them. Stereo applies only through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot.

The site describes two setups and does not say which one an equipment order uses. The hard-environments stack is a global-shutter 4K/60 head camera, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output. Which applies to these rows is Not stated.

A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the geometry sits in the same capture that shows the hands working. The hard-environments page states that stereo keeps working outdoors where active depth sensors struggle. Whether a given episode is outdoors or in a shed is Not stated for these rows.

A stereo pair adds depth, not a verdict. The site states no success, quality or skill label for operating or hitching, and none is claimed.

The rows give a natural cut: operate a tractor, mower or loader, attach and detach implements, drive, hitch, and operate the implement across the field. A spec that says "operate the tractor" gives one long action segment. A spec that names the steps gives a sequence you can slice by step, and the site does not set an episode length or say how many steps make up an episode.

Whether driving and operating are one segment or two is a spec decision, not a default. The site lists no control position, machine state or implement state label for these rows.

The site states that the hands occlude the work, that fingers disappear behind objects and that reach and grasp frames are where blur and occlusion show up. That is why each hand joint carries a visibility flag, why instance masks are recorded per frame and why wrist views are an option when a grip is lost from the head camera.

Cab glare, vibration, dust, outdoor lighting changes and implement occlusion are Not stated for these rows, and this page does not imply a fix. The site does state that glare-heavy and low-texture scenes make single-frame depth worse and that depth degrades on wet and reflective surfaces, with confidence maps masking low-confidence regions. A program that cares about time of day or a covered setting puts it in the spec and checks it on a Spec Pilot.

Equipment safety, operator licensing, certification and machine condition are Not stated for these rows, and this page makes no such claim and gives no operating guidance.

For every delivered episode the site states that faces and screens are blurred irreversibly, that a reviewer confirms no un-consented person is identifiable, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data and every episode links to a signed contributor release. The anonymization and face blurring pages describe the process in full. This is not legal advice, and the executed agreement controls.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. Firsthand offers MCAP delivery. Topic names, schemas, encoding and rates are Not stated on the site and are agreed in the skill spec.

Calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is rejected for a sync breach beyond the 2.0 ms ceiling for more than three consecutive frames, for the manipulated object out of frame for more than 15 percent of the episode, for more than 35 percent of depth pixels in the manipulation region being invalid, for an undocumented bystander or for a mismatch against the named skill spec. A reject log ships with every batch.

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. The site names no tractor or farm equipment dataset among them, so this work is custom collection, not an off-the-shelf purchase. The off-the-shelf stereo and hand-tracking video can still show the format before you write a spec.

It is custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. One row is Active collection and the other Ready to deploy, but the site does not say which speed applies to which, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Describe the steps and where each starts and ends, what counts as the object and as a finished step, the privacy rule for the space, and the streams and topic naming your reader expects. Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale.