Egocentric Pouring and Liquid Handling Video Data

A pour looks simple and is not. A person picks up the source vessel, positions it over the receiving one, tilts until the liquid starts to move, eases the angle as the level rises, stops, and sets both vessels down. The site describes the physics in the same terms it uses for kitchen work: pouring and decanting exercise controlled release and liquid-object contact. That is a motion where timing and angle matter at every moment, and a policy has to see how a person handles it.

A first-person camera records that sequence from the viewpoint a robot would act from. The operator leans in, watches the stream meet the opening and looks at the level before deciding to stop, so the video shows not only what was done but what could be seen when it was done. Hand pose and contact events record the handling, and action segments mark where each step starts and ends. This is the kind of evidence a manipulation policy can learn from without any robot being involved in the capture.

It helps to be exact about what this data is. It records a person pouring: the views, the handling and the order of steps. The site does not list a liquid-level or fill-state label, so that is not promised, and it does not state volume, flow-rate, spill, liquid-type or container-identity labels either. A buyer who needs any of them defines them in the written spec before capture begins.

Egocentric cooking video covers two-handed food preparation as a whole: knife work, pouring and decanting, opening jars, stovetop transfers, dishwasher loading and bimanual dough handling. Pouring appears there as one of several task categories. This page takes pouring and liquid handling as the organizing task family and follows it out of the kitchen, into the bar, the office, the care room, the wet lab and the farm.

The kitchen pretraining, food-service and hospitality pages are organized around a corpus, a sector and a type of venue. Each touches serving a drink or handling liquids somewhere in its scope, but none organizes the spec, the examples and the labels around the pour itself.

The site does not have a lab automation page. The catalog does list pharmacy and laboratory rows for pipetting, titration and liquid handling, and those rows are included here, with the catalog description, environment and status exactly as listed. The table near the end sets the neighbouring pages side by side.

The task catalog already lists pouring and liquid-handling work in several sectors. In food and beverage it includes professional barista workflows (pour drinks at the bar), pour-over coffee and cracking and whisking eggs, then pouring to a pan. In hospitality it includes pouring and serving a beverage; in an office, coffee machine operation; in a care room, serving a drink with a straw.

The catalog also lists a run of pharmacy and laboratory rows: liquid handling (measure and transfer liquids between vessels with pipettes, cylinders and dispensers), pipetting, pouring to a graduated cylinder mark, titrating a sample to a colour endpoint and a combined lab protocols row. The first table below lists every row.

Three further rows involve pouring or dispensing a material that is not a drink or a reagent: concrete work (mix, pour, screed and finish concrete into forms), feeding and watering livestock (carry feed buckets to troughs, pour feed and fill water troughs with a hose) and applying an adhesive bead (dispense a bead on a path). They are listed because the catalog states them, not because the site calls them liquid handling, and they are marked as such in the table.

The catalog places these tasks in ten environment types, from a cafe and bar to a wet lab and a farm; the first table names each one. Every task row carries a status that says how soon capture can begin.

The site defines those statuses plainly. Ready to deploy means contributors and site access are set up and capture starts once a spec is signed. Active collection means recording is running now, so a spec can join an existing capture stream. Available on request means the work is scoped and recruited to spec when a buyer asks for it. In the current catalog, feeding and watering livestock is marked Active collection. The other fourteen pouring and liquid-handling rows are Available on request, and none is currently marked Ready to deploy.

The site does not say that any of this capture takes place inside a particular customer facility, and this page does not claim it. The catalog names environment types, and where a collection happens is agreed in the spec.

The written spec is the contract, so it is worth being specific about the parts of pouring the site leaves to the buyer. The site already says what a kitchen pour spec should cover: the vessels, the sub-steps and what counts as a completed pour. Those three are a good starting list for any pouring task.

Name the vessels in scope, such as cups, bottles, jugs, pans, graduated cylinders or tubes, and which one is the source and which is the receiver. Name the liquids at the level your policy needs, because water, milk, a viscous liquid and a reagent are handled differently, and the site does not label liquid type by default. Name the tools around the pour, which the catalog mentions as a grinder, a filter, a coffee machine, pipettes, cylinders, dispensers and a hose.

Name the start state and the end state. A pour to a measured mark and a pour until a vessel is empty are different goals, and the accept criteria should say which one counts as a finished episode. Finally, name the capture mode: a head-mounted view of real work on site, or a staged set with handheld grippers for demonstrations that must map onto a robot end effector.

The catalog names two capture modes. In the first, a head-mounted phone records the first-person view of real work on site, and a multiview rig adds synchronized third-person angles when the spec asks for them. In the second, a staged working set is recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector.

A head-mounted camera keeps the vessels, the hands and the surface in frame from the viewpoint a manipulation policy will see from, with a wrist RGB stream alongside it for close-range detail. Depth, 3D hand pose, contact events and action segments are aligned to video on one hardware clock, synced under 2 ms. Distance-invalid pixels beyond the depth sensor reliable range are flagged rather than interpolated.

The site does not state a camera count, resolution, frame rate or lighting setup specific to pouring, so those are agreed in the written spec rather than assumed. The site also does not state how depth behaves on clear liquids, glass or reflective surfaces, which matters for any task where the vessel or the liquid is transparent, so that is something to raise in the spec and test in a Spec Pilot.

Each episode follows the same episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. The cooking page also states per-frame instance masks for deformable and liquid ingredients, depth confidence maps and an episode-level steam flag; the site does not state those specifically for pouring tasks outside cooking.

For pouring, hand pose tracks the operator through the lift, tilt, hold, return and set-down motions, contact events mark when fingers make and break contact with a vessel, and action segments mark the start and end of each step. The cooking page already gives the example that a segment might be one cut, one pour or one transfer, as written in the skill spec. Because the labels share one clock with the video and depth, a step boundary lines up with the exact frame where it happens.

Object labels use the terms your spec sets: the skill spec decides which object classes are in scope before capture, so labels follow the vocabulary your training stack expects rather than a generic taxonomy.

Fill-level and fill-state labels, volume and flow-rate values, spill labels, liquid-type labels, container-identity labels and task-completion labels are not stated anywhere on the site as standard labels. The site says so directly for the kitchen case: no liquid-level or fill-state label, and no force, tactile or thermal data is listed. Some catalog task names describe dispensing exact volumes or titrating to an endpoint, but those describe what the operator does, not a measurement delivered with the episode.

A per-episode success or failure label is also not stated. If your policy needs any of these, they are defined in the written spec as annotation depth, and the quote covers them before capture begins. The page does not promise them by default.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Contributors are the people who perform the pouring tasks, so their release covers the footage captured of their hands and workspace.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data, so a buyer can see what was blurred and when. That matters at a bar, an office coffee point or a lab bench, where customers, coworkers, point-of-sale screens and instrument displays can enter the frame.

Bottle labels, reagent labels and container markings are not listed as blur categories on the site, so this page does not claim they are blurred. Pouring puts printed labels in close view by nature. If label text in frame matters for your use, raise it in the spec so the handling is agreed before capture rather than assumed afterwards. The catalog also notes that the pharmacy and laboratory work requires regulatory compliance, and this page makes no compliance or certification claim of its own.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review, which suits visually checking a pouring episode before writing a loader.

Converter source is readable, so you can retarget the schema to your own loader, and the episode schema page lists the fields. Pouring episodes use the same schema as every other Firsthand collection, so a loader that already reads another Firsthand dataset reads a pouring dataset without changes.

You describe the vessels, liquids, stations, tools and end states your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price against that document before any capture begins.

The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization and licensed delivery. Quality is measured against signed accept criteria, billing is per validated hour, a reject log ships with every batch, and rejected items are recaptured at our cost.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front. The catalog status on each task row tells you which case you are in.

On-demand top-ups run against the same spec and accept criteria after each training run, so new pouring data stays consistent with what you already hold. A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale volume.

Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. The license is documented alongside the consent chain, so a policy trained on the data can go into a product.