Egocentric Livestock Handling Data

Egocentric livestock handling data is first-person video of a person doing animal-care work on a farm, recorded from their own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see, so the frames show what the hands are doing with the buckets, hoses, forks and shovels. The same idea sits behind any livestock handling robot training data or animal husbandry video dataset request, applied here as a custom collection scoped to your tasks.

The word handling needs care. On this site the farm rows describe feeding, watering and pen cleaning. They do not describe restraining, moving or examining an animal, and the farm environment has no row for it. This page covers the rows that exist and says plainly where a request goes beyond them.

The data is human capture: a paid, consented contributor wears the rig and performs a task from a written skill spec. It is not robot or teleoperation data. The farm and field page covers the wider environment, and the veterinary clinic page has its own scope.

The task catalog lists the Farm & Field Operations environment under the Agriculture & Grounds sector. It has 21 rows, and three of them are about animals directly. "Feed and water livestock using feed buckets and hoses" is Easy and Active collection: carry feed buckets to troughs, pour feed and fill water troughs with a hose. "Feeding & Watering Rounds" is Easy and Ready to deploy: walk the pens or fields delivering feed and topping up water for the animals. "Animal Pen Cleaning & Waste Removal" is Easy and Ready to deploy: muck out pens with fork and shovel, remove waste in barrows and lay fresh bedding.

A fourth row, "Egg Collection & Candling", sits nearby: collect eggs from the nests, check them against a light for defects and pack them. The catalog does not call it livestock handling, so the coverage table lists it as an adjacent row rather than counting it.

The site states no animal types, herd or flock sizes, farm types, seasons or regions, or cycle times.

No. No Farm & Field Operations row names restraining or lifting an animal, herding or driving animals, milking, shearing, hoof care, health checks or treatment. The coverage table below sets each of these against what the catalog actually lists.

One row sounds close but belongs elsewhere. "Animal handling" is a Veterinary Clinic row, described as lifting, holding and restraining animals on the table during examination. It has its own page, and this page does not count it as livestock work. If a program needs a task the catalog does not name, it is written into the skill spec, and whether it can be captured is confirmed when the spec is quoted rather than assumed.

Each attempt is delivered as one episode. On this site the standard episode carries action segments, 3D hand pose for both hands with a per-joint visibility flag, contact events that mark when a hand meets an item, an objects list, one task instruction, and per-frame instance masks stored at the depth resolution of 848×480. The capture setup table below lists the stated streams and sync figures.

The catalog lists one capture mode for every farm and field row: a head-mounted phone with a multiview rig. Force and tactile readings are not among the listed streams.

Action segments divide an episode into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For livestock work, a segment might be one bucket carried, one trough filled or one barrow emptied, as written in the skill spec. The site lists no label for the outcome of a step, such as whether a trough was left full or a pen was left clean.

Validation runs against signed accept criteria written into the spec before capture, and a reject log ships with each batch. A human reviewer checks every episode. Only footage that clears the criteria is billed as a validated hour.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata, with readable converter source alongside. The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity available as a separate term. The executed agreement controls.

The catalog describes the work in repetitive, physical steps: carrying buckets, pouring, filling a trough from a hose, mucking out with a fork and laying bedding. Feed, water and bedding are loose or flowing materials, so the same step never looks quite the same twice. Buckets, hoses and forks also cover the fingers. That is why hand pose carries a per-joint visibility flag, why contact events mark when the hand meets the item, and why masks are recorded per frame. The challenge table below maps each difficulty to the stream or label this site states.

Animal behavior, light and weather are separate problems. The site lists an IMU and body pose as streams but states nothing about animal movement, handling technique, lighting or weather, so the table says so rather than implying a fix.

A robot foundation model paper on arXiv puts human videos at the base of its data pyramid and says they have no explicit action labels, so it learns latent actions. That is robot-model research, not Firsthand data, and Firsthand has no relationship with its authors.

The catalog describes the head-mounted phone as recording the first-person view of real work. The site also states that Firsthand does not capture at a customer’s own facility. Which farm a program uses is agreed in the skill spec.

Every contributor signs a written release, and each delivered episode links to it through consent artefact IDs. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers. A human reviewer confirms that no un-consented person is identifiable in any frame, and the anonymization record ships with the data.

Safety certifications, protective equipment, machinery safety and animal welfare are not stated for this environment, and this page makes no such claim.

From signed spec to first validated batch, the published timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. The catalog marks the feed-and-water row Active collection, meaning recording is already running and a spec can join it. The feeding rounds and pen cleaning rows are Ready to deploy, meaning contributors and site access are already set up. Which range applies is confirmed when the spec is quoted.

Describe the animal-care tasks and their order, the buckets, hoses and tools involved, what counts as a finished step, and the failure or variation cases your pipeline is missing. If you need something beyond the three rows, such as restraint or herding, say so up front. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins. Our skill spec guide walks through how to write one.

Most programs begin with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales. Collections are billed per validated hour. Counts of sites, episodes and hours are set in the skill spec.