Stereo Greenhouse Potting and Pruning Data

It is first-person video of a person doing bench work in a greenhouse or nursery, recorded with two calibrated cameras and with hand pose and sensor streams alongside. The tasks are the seven rows the site’s catalog lists in Greenhouse, Nursery & Garden Centre: filling a pot, planting and watering; cutting stems at the node with shears; placing seeds into tray cells; lifting and replanting seedlings; staking and clipping a stem for support; weaving stems into a trellis; and trimming spent growth and deadheading. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a multiview rig with handheld grippers as the capture setup, which the site describes as a staged working set recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. Stereo applies only through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Seven, all in Greenhouse, Nursery & Garden Centre under Agriculture & Grounds, in the catalog data file lib/commercial-catalog-data.ts. The catalog lists one example set for the environment: "Greenhouse / grow-bench set: pots, trays, plants, shears, stakes, crates". "Pot plants" is described as filling a pot, planting and watering, is rated Easy and is listed as Available on request. "Prune with shears" is described as cutting stems at the node with shears, is rated Medium and is listed as Available on request. "Seed a tray" is described as placing seeds into tray cells, is rated Medium and is listed as Available on request.

"Transplant seedlings" is described as lifting and replanting seedlings, is rated Medium and is listed as Available on request. "Stake + clip plant" is described as staking and clipping a stem for support, is rated Medium and is listed as Available on request. "Trellis training" is described as weaving stems into a trellis, is rated Medium and is listed as Available on request. "Trim + deadhead" is described as trimming spent growth and deadheading, is rated Medium and is listed as Available on request. All seven list a multiview rig with handheld grippers as the capture setup. The site says nothing further about what Available on request means in volume, hours or delivered stereo footage, so this page claims none. The first table below reproduces the rows as the catalog states them.

The site defines Available on request as scoped and recruited to spec when a buyer asks for it, so recording is not already running for these rows. That differs from Active collection, which the site defines as recording that is running now, so a spec can join an existing capture stream. None of the seven rows carries either of those labels, and none is listed as Ready to deploy.

In practice a stereo greenhouse order starts from your written spec rather than from an existing stream. The site states no delivered hours and no delivered stereo footage for any of the seven rows, so this page makes no such claim. What a given order delivers, and when, is quoted against your spec.

Two rows sit next to this work. "Harvest ripe produce" is in the same Greenhouse, Nursery & Garden Centre environment, is described as picking ripe produce by twist or snip, is rated Medium and is listed as Available on request, with the same multiview rig and handheld grippers setup. "Pruning" is in Farm & Field Operations, is described as cutting back dead and overgrown branches with shears and loppers to shape the plant, is rated Medium and is listed as Active collection, with a head-mounted phone or a multiview rig as the capture setup.

Both are context only: this page builds nothing on them. Harvesting is treated on the stereo crop harvesting page, and the pruning rows together with the farm row are treated on the egocentric pruning and trellis training page. Keeping them out of this page keeps its status and capture statements true to the seven rows named above.

Three egocentric pages already cover these rows by task. The plant potting and nursery page covers "Pot plants" and "Transplant seedlings"; the seed tray page covers "Seed a tray" and "Transplant seedlings"; the pruning and trellis page covers the four greenhouse pruning rows and adds the farm row "Pruning". Each treats capture as egocentric and delivers in RLDS, LeRobot or WebDataset. This page spans all seven greenhouse rows in one stereo treatment, and goes further into the stereo pair, the MCAP container and the failure modes at a grow bench.

Two stereo pages sit nearby. The stereo crop harvesting page covers four harvesting rows, three of them Active collection, and shares this page’s stereo and MCAP treatment, but its task is picking and cutting produce into crates. The stereo landscaping and groundskeeping page covers one Active collection row for mowing, trimming, planting and grounds maintenance on commercial property, a different environment from a greenhouse bench. None of those five pages centres stereo capture of the seven greenhouse potting, pruning, seeding, transplanting, staking and trellis rows. The comparison table below sets them side by side.

Bench work is about where the hand, the tool and the plant sit relative to one another: how far the shears are from the node, where a seed sits against a tray cell, how a stem lies against a stake or a trellis wire. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the plant, its container or support and the hands share a frame.

The site gives no depth accuracy figure and no label for a cut at the node, a seed placed in a cell, a seedling settled in its new container or a stem secured to a support, so whether those outcomes are resolvable in depth is Not stated. Capture at your own facility is listed as not offered.

The catalog states one capture setup for all seven rows: a multiview rig with handheld grippers. The site describes two stereo setups and does not say which one a greenhouse order uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.

Which rig, resolution and streams a potting, pruning, seeding, transplanting, staking or trellis order uses is Not stated on the site, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. The catalog states that a multiview rig adds synchronized third-person angles when the spec asks for them, and wrist cameras give a closer view of the hands. The site does not state a greenhouse lighting setup, so lighting is Not stated and is agreed in the skill spec.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For greenhouse bench work the site states hand pose, depth recovered from the stereo pair, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate. The only language data tied to Firsthand capture is one task instruction per episode.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Hand occlusion by plant material and tools is the first risk. The site states that fingers disappear behind objects and that plants occlude the hands, and a hand closing shears on a stem, holding a seedling by its root ball or weaving a stem through a trellis is partly hidden from the head camera. The visibility flag on each joint is the answer the site gives for hidden fingers, contact events mark when a hand or the shears meet the plant, and a spec can ask for wrist views so a grip the head camera loses is still recorded. The site states no stem, leaf or seed label, so what counts as one cut, one seed placed or one stem secured is written into the spec.

Thin, deformable plant material, cluttered benches and variable light are the second. The site names reflective surfaces and clutter as cases where a single camera struggles in hard environments, and notes that wet and reflective surfaces degrade active-stereo depth in its kitchen dataset. Whether wet leaves, damp potting media, small seeds in tray cells or a bench crowded with pots, stakes and crates behave the same on a stereo pair is Not stated, and so is the effect of greenhouse light. The site ships per-frame depth confidence maps for its kitchen captures, and whether a greenhouse order ships them is Not stated, so ask in the spec.

Calibration is the third, because stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it or an undocumented bystander appears, and a reject log ships with every batch. Force and tactile readings are not among the listed streams.

Bench work chains steps that look alike on camera: filling one pot and then the next, placing one seed and then the next, lifting one seedling and replanting it before moving to another, trimming one stem and then another. Action segments carry start, end, verb and noun, so the spec decides where one segment ends and the next begins, and which object is the noun.

The site states no episode length, no segment count and no per-plant label for these rows, and the catalog does not split any row into finer steps. Whether an episode is one pot, one tray, one seedling, one plant or a full bench sequence is set per program. Put that boundary and the verbs you expect into the skill spec, and check them on the Spec Pilot before you scale.

For every delivered episode the site states that faces, plates and screens are detected and irreversibly blurred at ingest, that a reviewer confirms the result before anything ships, that any identifiable un-consented person goes to human review, and that footage that cannot be de-identified is discarded. Every episode links to a signed contributor release through a documented consent chain, and the consent record and the anonymization record are delivered with the data.

A greenhouse or nursery can have coworkers in frame, and who may be in frame is agreed in the spec. The site states no grower, nursery or client name, no plant species beyond the plants named in the rows, and no plant count, yield, task time, throughput or error rate for these rows. Each of those is Not stated. This page makes no yield, plant-health, chemical, pesticide, fertilizer, safety or certification claim, and no rule for data stored on a device is stated.

Off-the-shelf covers what is already in stock: the stereo category lists the stereo datasets and hand-tracking video, and only dataset 06 is stated as MCAP. This page does not claim an off-the-shelf dataset of potting, pruning, seeding, transplanting, staking or trellis work, so this is custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.

Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The site does not say which speed applies to a stereo greenhouse order, so ask when you request the quote. Collections are billed per validated hour, rejected items are recaptured at our cost, and on-demand top-ups run against the same spec and accept criteria after each training run.

Describe the bench sequence and its order, the plants and containers involved, the tools and supports, what counts as a finished step, and the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.