Egocentric Plant Care and Gardening Video Data

Egocentric plant care and gardening video data is real-world, first-person video of a person looking after plants: filling a container and watering, potting and planting, pruning, transplanting seedlings, staking a stem and picking produce by hand. It is recorded from the operator viewpoint, with depth, 3D hand pose, contact events and frame-accurate action segments aligned on one hardware clock.

Plant care asks a lot of a manipulation policy because the objects are soft, irregular and different every time. No two stems, pots or soil beds look alike, and the work mixes gentle grasping with tool use and with moving through a space. A first-person view keeps the plant, both hands and the container or tool in the same frame, which is the viewpoint a robot arm with a head or wrist camera would act from.

This page treats plant care and gardening as a task family in its own right. The household chores dataset shows plant watering as one of its sample clips, the tool-use page covers hand tools in general, and the pouring page covers liquid handling. Here the task centres on the plant and the work done to it.

The site states plant care in two places. The task catalog lists it under its Agriculture & Grounds sector, in a Farm & Field Operations environment and in a Greenhouse, Nursery & Garden Centre environment, with rows for watering, potting, pruning, planting, transplanting, staking, trellis training, trimming and deadheading, seeding trays, filling hanging baskets and picking by hand. Separately, the site publishes first-person sample clips of watering houseplants at home and of working soil in a raised bed.

Skill ratings run from Easy for potting, hanging baskets and hand harvesting to Medium for pruning, planting, transplanting and staking. Statuses run from Available on request in the greenhouse and nursery environment to Active collection and Ready to deploy in the farm environment.

A custom spec is scoped from that vocabulary. You describe the plants, containers, stations and sequence your policy needs, and the written spec covers them, with the catalog rows as a starting point rather than a fixed menu. The tables below list what the site states and the rows with their skill rating and status.

Watering is the best-evidenced plant task on the site because it is published as footage, not only as a catalog line. Two sample clips show an operator filling a container and watering houseplants: one at a kitchen sink with the container carried across the room to plants by a window, and one in a different home where a vessel is lifted from a desk, carried across the room and used on a plant beside a bookcase.

In the catalog, watering appears inside rows rather than as a standalone row. The Greenhouse / Nursery Tending row is described as watering, potting, pruning and pest-checking plants on greenhouse and nursery benches, and the Planting row ends with covering and watering the plants. Irrigation rows cover moving and connecting lines and opening valves, which is a different job from hand watering.

The site states no watering volume, can or hose type, plant type or number of plants per episode. Those are agreed in the written spec. The pouring and liquid handling page covers the general hand motion of tilting a container.

The Greenhouse, Nursery & Garden Centre environment has a Pot plants row, described as filling a pot, planting and watering, and a Fill hanging basket row, described as filling and hanging a planted basket. Both are rated Easy and are Available on request. The greenhouse and nursery tending row in the farm environment includes potting as one of its steps and is Medium and Ready to deploy.

The motion is a short chain with a clear order: fill, set the plant, firm, water. Action segments are useful because the boundaries between those steps are the moments a policy has to sequence correctly. A spec can ask for the whole chain or isolate one step.

The site does not list repotting as its own row and states no pot size, pot type or soil type. A spec that needs repotting, a particular container or a particular medium defines it up front.

Several rows cover cutting back plants. Pruning is described as cutting back dead and overgrown branches to shape the plant. A greenhouse row is described as cutting stems at the node, and another as trimming spent growth and deadheading. A separate row covers cutting hedge tops and sides, removing thick stems and clearing the cuttings. These are rated Medium; the farm rows are Active collection and the greenhouse rows are Available on request.

Cutting is a precision task: choose the spot, position the tool, close, and move the cutting away. The hand pose and contact events record the choice and the cut, and a wrist-mounted camera is available for close-range detail. The tool-use page covers the general pose and contact detail behind hand tools.

The site states no plant species, no stem diameter and no cut-quality label, and it names no tool brand. Those, if needed, are defined in the written spec.

Planting is described in the farm environment as sowing seeds or setting plants in prepared soil at spacing, covering and watering them, and is rated Medium and Active collection. A transplanting row is described as taking seedlings from trays, digging, and planting them at spacing along the row, also Medium and Active collection. A planting and transplanting row is described as digging holes or furrows, setting seedlings in place and firming the soil around them, Medium and Ready to deploy.

The greenhouse and nursery environment adds Seed a tray, described as placing seeds into tray cells, and Transplant seedlings, described as lifting and replanting seedlings. Both are Medium and Available on request. Together they span ground planting, tray work and bench work.

The site states no seed size, spacing value, row length or tray format. A spec sets them when the policy depends on them.

Three greenhouse and nursery rows cover supporting growth: Stake + clip plant, described as staking and clipping a stem for support, and Trellis training, described as weaving stems into a trellis. Both are rated Medium and are Available on request. They sit in the staged capture mode described below, in which an operator works a staged set with handheld grippers so the demonstrations map onto a robot end effector.

These are bimanual tasks with a clear plan: hold one part, move another, fix it in place. Contact events mark when the fingers make and break contact with a stem, and 3D hand pose carries the grip while the other hand works.

The site states no stem count, support type or clip type. Those are defined in the spec.

Hand harvesting is stated several ways: picking ripe fruit or vegetables by hand into crates, cutting produce from the plant and sorting it, and a greenhouse row for picking ripe produce by twist or snip. A broader row describes planting, tending, harvesting and handling produce in commercial growing operations. Harvesting rows are Easy or Medium and are Active collection or Available on request.

Grading, packing and crate hauling are neighbouring rows rather than plant care, and this page uses them as context. They are covered by the box and parcel handling and object rearrangement pages, not here.

The site states no crop type, ripeness label, yield figure or pick rate, and no throughput claim is made here.

The catalog has no standalone weeding row. Soil work is evidenced by the raised-bed clip, in which a gloved operator breaks up soil with a small hand tool between seedlings. Pest-checking is stated as part of the greenhouse and nursery tending row. A landscaping row describes mowing, trimming, planting and clearing grounds and paths on commercial property.

This is the honest edge of what the site states. A buyer who needs weeding, soil preparation or a pest-check label asks for it in the spec, and the quote covers it before capture begins. The page does not promise any of these as a default delivered task.

The site publishes three first-person clips relevant here. Watering plants at home, 1, is 20 seconds from a fisheye camera at 360 by 360 and covers filling a container, crossing the room and watering. Watering plants at home, 2, is 10 seconds at 960 by 540, performed by a different person in a different home. Working soil in a raised bed is 61 seconds from a wide-angle camera at 640 by 360.

The site draws two lessons from them. The same watering task across two households is described as the difference between a policy that generalizes and one that memorizes a single kitchen. The raised-bed clip is the first outdoor capture in that set, with direct sun, hard shadows and gloved hands that remove the usual skin-tone hand cues, so a model has to track the tool.

The site also states a limit: the raised bed is a home bed, not commercial growing, and the seasonal agriculture catalog line is a separate thing not evidenced by that clip. The clips are samples, not a plant care dataset with a stated hour count, and none is stated here.

The catalog places plant care in a farm and field environment, a greenhouse, nursery and garden centre environment, and a landscaping and groundskeeping environment, all within its Agriculture & Grounds sector. The site publishes home clips for watering and for a raised bed. Which of these is the right setting depends on whether your policy is meant for a home or a commercial growing space.

The site defines the statuses plainly. Ready to deploy means contributors and site access are set up and capture starts once a spec is signed. Active collection means recording is running now, so a spec can join an existing capture stream. Available on request means the work is scoped and recruited to spec when a buyer asks for it, which is the status of every greenhouse and nursery row.

The site does not state capture at a customer own farm, nursery, garden or facility, and it names no grower or brand. The share of indoor versus outdoor episodes is not stated. Environment, access and schedule are agreed in the written spec.

The catalog names two capture modes for these rows. In the first, a head-mounted phone records the first-person view of real work on site, and a multiview rig adds synchronized third-person angles when the spec asks for them. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector.

Every greenhouse and nursery row uses the staged mode, while the farm and landscaping rows use the head-mounted mode. A wrist-mounted camera is available alongside the head camera for close-range detail. Depth, 3D hand pose, contact events and action segments are aligned to video on one hardware clock, synced under 2 ms.

The site does not state a camera count, resolution or frame rate specific to plant care, apart from the sample clip resolutions above, so those are agreed in the written spec rather than assumed.

Each episode follows the same episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. For its household chores dataset, which carries the watering clips, the site also states per-frame instance masks for deformable objects.

For plant care, hand pose tracks the operator through the grip, the cut and the placement, contact events mark when the fingers make and break contact with a stem, a pot or a container, and action segments mark the start and end of each step. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens.

Object labels use the terms your spec sets. The skill spec decides which object classes are in scope before capture, so labels follow the vocabulary your training stack expects rather than a generic taxonomy.

Plant species, soil or pot types, tool specifications, throughput, force or tactile data and a task-completion label are not stated anywhere on the site as standard fields. Weeding as a standalone row, repotting as a standalone row, and the split between indoor and outdoor episodes are not stated either.

If your policy needs any of these, they are defined in the written spec as annotation depth, and the quote covers them before capture begins. The page does not promise them by default.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review. The format table below sets each against what its official documentation states.

Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields. Plant care episodes use the same schema as every other Firsthand collection, so a loader that already reads another Firsthand dataset reads this one without changes.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Contributors are the people who look after the plants, so their release covers the footage captured of their hands and workspace.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data. Home footage can show bystanders and reflections, and the site states that those categories are covered.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. The site names agriculture among its thin coverage areas, with construction, clinical and hospitality, where a first batch typically takes three to four weeks because vetting operators there is the bottleneck. Plan on that range for commercial growing rows, and note that every greenhouse and nursery row is Available on request.

The site does not state how deep coverage is for home plant care, so that timeline is quoted against your spec. On-demand top-ups run against the same spec and accept criteria after each training run, so new plant care data stays consistent with what you already hold. A batch size and a later-batch turnaround are not stated and are quoted too.

You describe the plants, containers, stations and sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

A Spec Pilot, a small and fast first batch, can prove the spec and the accept criteria before you scale volume. The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization and licensed delivery. Billing is per validated hour, a reject log ships with every batch, and rejected items are recaptured at our cost.

Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. The license is documented alongside the consent chain, so a policy trained on the data can go into a product.