Egocentric Dishwashing Video Data for Robot Training

Dishwashing is the work of taking used dishes, utensils and cookware from a dirty state to a clean, stored state. A person sprays off or hand-washes the wares, racks them, runs a dishwasher or lets them dry, then puts them away. Kitchen clean-up is the neighbouring work of wiping and resetting the surfaces around them.

Dishwashing data is a recording of that work together with the signals a policy learns from: where the hands go, which ware is being handled, when contact is made and broken, and how the task divides into steps. This page treats dishwashing and clean-up as a task family with its own capture requirements, not as a footnote to cooking.

The kitchen pretraining page is organized by training stage: broad capture of food prep, cleaning and utensil use across many kitchens, for the base layer of a model. The cooking page is organized by one activity, a recipe as a long chain of dependent steps. The food-service page is organized by industry. None of them is organized around the clean-up chain itself.

This page is. It covers the wash, rack, run, dry and put-away sequence and the wet-surface conditions that come with it. A dishwashing program can feed kitchen pretraining, and cooking episodes end with dishes that need washing, but the spec, the sensing concerns and the failure modes here are specific to the clean-up task family.

The site describes the family in three places. The kitchen dataset listing names loading and unloading a dishwasher and wiping and resetting a work surface among its skills. The kitchen pretraining page describes cleaning, loading and unloading as part of kitchen capture. The task catalog behind the environments hub lists dishwashing rows in a working kitchen set, a restaurant kitchen and a hospitality service set.

The task-family table below gives each one with the catalog wording. A catalog listing is a scoping aid, not a statement that the task has been captured, and the catalog does not list rinse-only, soak or pot-scrubbing tasks as separate rows. Anything beyond the listed rows is defined in the written skill spec.

Each step in dishwashing depends on the one before it. Wares cannot be racked until they are washed, the dishwasher cannot run until the rack is loaded, and clean wares cannot be put away until they are unloaded. A policy trained on isolated grasps sees none of that order.

That is why a dishwashing spec names the full arc and why action segments matter. Segments with a start and end to the frame, drawn from the site’s stated action vocabulary, split a long episode into its wash, rack and put-away parts. Contact events mark each pick and release inside those parts, and hand pose shows how each ware was held.

Water is the defining condition of dishwashing. The site states that wet and reflective surfaces, such as a rinsed pan, a wet countertop and a steel sink, degrade active-stereo depth. Kitchen captures therefore ship per-frame depth confidence maps alongside the depth stream, so you can mask low-confidence regions instead of trusting them.

Steam is labelled as an episode-level flag where it occurs, because it disrupts depth for 2 to 6 seconds at a time. An episode with more than 35 percent invalid depth pixels in the manipulation region goes to the reject log and is recaptured at our cost. The wet-conditions table lists each condition with what ships. Whether the rig itself is water-protected is not stated, so any splash or handling requirement is confirmed per program.

Dishes are slippery when wet and can be dropped, and the site lists drops, spills, mis-grasps and recoveries among the failure cases in its kitchen dataset conditions. The labels the site states for handling are contact events on grasp and release and 3D hand pose with per-joint visibility. Those show when a ware was gripped and let it be released again, and how the fingers held it.

A fragile-object label, a breakage label, force data and tactile data are not stated. If your policy needs to know that a glass is more delicate than a plate, that is defined in the spec as a ware category and confirmed per program, not assumed from a stated field. The failure and recovery page covers capturing drops and corrections deliberately.

The head camera keeps the sink, rack and the hands in view from the actor’s own viewpoint, and the two wrist cameras give a close view of each ware as it is handled. Depth carries distance to the wares and stations, with a confidence map for the wet regions. IMU data records head motion between the sink, the dishwasher and the cupboard on the same clock.

Hand pose, body pose, contact events, instance masks and action segments are the labels. The streams-and-labels table gives each with its stated spec. Head or camera 6-DoF pose and a SLAM trajectory are not stated, so the page does not rely on them, and the site states no robot or teleoperation capture for this program.

A skill spec is the written contract the capture is run against. For dishwashing it should name the task and its end state, the wares involved, the stations used and the kitchen type. The wares matter most: plates, bowls, glasses, cutlery, pans and containers behave differently when wet, and a dataset that is all plates teaches little about a glass.

It should also say whether the work is hand-washing, loading a dishwasher, unloading one, or the whole chain, and where clean wares end up. The site states that a spec is defined before capture and that an episode that does not match the named spec goes to the reject log and is recaptured at our cost. A fixed object list and a scene-reset protocol are not stated, so both belong in the spec.

The site lists both. The kitchen dataset covers a domestic kitchen, a shared-house kitchen and a small commercial prep line. The task catalog separates a working kitchen set, where the listed tasks are loading and unloading a dishwasher and washing and drying dishes by hand, from a restaurant and commercial kitchen, where they are operating a dish pit and washing and racking dishware with a commercial dishwasher and spray hose.

The choice depends on the deployment. A home robot meets a sink, a household dishwasher and a mixed set of wares. A food-service robot meets racks, a spray hose and repetitive volume. The catalog also lists a capture field for these rows, which is a scoping field, so how a given task is recorded is confirmed in the spec. The stated rig on this site is worn by the contributor.

Kitchen capture happens in real homes or shared prep spaces, which raises the bar on consent and anonymization above a controlled workshop. Every participant signs a written release granting commercial training rights before capture begins, and is paid a published hourly rate. Faces and licence plates of bystanders are blurred, audio is scrubbed of identifying speech, and clips where a bystander cannot be de-identified are discarded rather than shipped.

You receive a perpetual, irrevocable, buyer-owned commercial license and the consent artefacts for the batch. Dishwashing is a close-range, hands-down task, so the usual privacy exposure is a household member or colleague passing through the frame, which is what the undocumented-bystander criterion catches.

Data ships on synchronized streams in the formats the site states: RLDS, LeRobot and WebDataset, with the episode schema documented. Scoping starts from the written spec and the kitchens, wares and tasks you need. A first validated batch lands in a median of 48 hours from a signed spec where coverage is deep, and in three to four weeks where it is thin.

Billing is per validated hour, quoted against the spec before capture, and rejected items are recaptured at our cost. The site states no hour count, throughput or customer for dishwashing, and none is implied here. The guide to scoping a custom collection explains how to turn a task list into a quote.