Egocentric Sushi and Cold Food Prep Data

Egocentric sushi and cold food prep data is first-person video of a person preparing food by hand, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see at a counter, so the frames show what the hands are doing to the greens, the toppings, the plate and the garnish rather than a distant view of someone standing near a kitchen. On this site it is a custom collection: you name the task and the setting, and capture is run to a written skill spec.

It helps to separate what the site states from what it does not. The catalog lists a row for assembling a salad, described as "Tear greens, add toppings, dress, and toss a salad," and a row for plating, described as "Plate a portion and add garnish." Both sit in one environment, a Working Kitchen Set for food prep. The catalog does not list a sushi row, a row for rolling rice or fillings, a cold station row, or any row that it labels as cold. This page therefore does not present sushi capture as an existing task. Whether a particular sushi step can be captured is a skill-spec question to settle in a Spec Pilot rather than an assumption, and until it is settled the honest answer on this site is "Not stated".

People reach this topic through several phrasings: sushi prep robot training data, a cold kitchen video dataset, egocentric rolling and plating data, or custom food prep data collection. They describe one request from different angles. Under each of them the work is the same: write down the task, collect it on demand, and receive validated episodes with the labels a training pipeline expects. The one phrase the catalog cannot answer directly is "rolling". The only rolling row on the site is for dough, and it belongs to baking, not to sushi.

This page overlaps on purpose with several neighbours. The sandwich and salad assembly page covers the salad row and an order assembly row. The meal plating and serving page covers plating, portioning and serving across the catalog. The cutting and food prep page covers peeling and chopping, and the baking and dough page covers kneading and rolling dough. The two rows this page describes are the same rows those pages describe. What it adds is the framing of cold prep and sushi as a request, and a plain account of what the site does not state about it.

Two rows of the site’s commercial task catalog describe a food prep step that fits this topic. The first table below reproduces them exactly as the catalog states them, with the environment, skill level, capture setup and status. Both sit in the Food & Beverage sector, in the environment called Working Kitchen Set (food prep), whose listed examples are "Stovetop, counter, sink, dishwasher, utensils, food props." That environment lists fourteen task rows in total, and most of the others are covered on other pages.

Assembling a salad is rated Medium and plating with a garnish is rated Easy. Both are listed as Available on request, and both are captured with a multiview rig and handheld grippers. The site does not describe the grippers further, so this page does not either. The catalog does not label either row as cold, so "cold" here is the topic of the page and not a property the catalog states for these rows.

The second table lists the neighbouring rows that were deliberately left out, with the page that covers each. Order assembly, which assembles burgers, wraps and sides, sits in a fast food kitchen and is covered on the sandwich and salad page. Banquet plating portions hot dishes, so it is not cold prep and is covered with plating. Peeling and chopping vegetables belongs to the cutting page, kneading and rolling dough to the baking page, and the garnished bar drink to the beverage pages. None of them is a sushi row either.

The statuses are the catalog’s own. The catalog is a list of what is captured or available, not a ceiling on what can be briefed. A dish, an ingredient, a step or a setting outside these rows is a skill-spec question rather than an assumption. The site does not describe an ingredient list, a portion size, a plating style or a task duration for either row, so none is stated here.

Each attempt is delivered as one episode. For the instrumented tier the site describes a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The catalog lists the same capture setup for both rows, a multiview rig with handheld grippers. Which tier a program uses is set in the skill spec.

Hand pose is delivered for both hands as 21 joints per hand in three dimensions, with a per-joint visibility flag for occluded fingers. Food prep is close work: a hand closing around a handful of greens or a plate hides fingers, and the flag marks the joints that were not visible instead of presenting a guess as a measurement. Contact events mark when a hand meets the item.

Every stream is hardware-triggered onto one PTP clock domain, so depth, motion and hand pose line up with the same video frame rather than being approximated afterwards. The rig is calibrated twice per session, and an episode is accepted only if RMS reprojection stays under 0.28 px.

The only language field the site ties to Firsthand capture is one task instruction per episode. The site does not list a temperature, weight or food-state stream for these rows. This is human first-person capture, not robot or teleoperation data.

Inside each episode, action segments divide the timeline into spans, each carrying start_ns, end_ns, verb and noun. For this task family a segment can be one step as written in the skill spec, such as tearing the greens, adding toppings, dressing and tossing, or plating a portion and adding the garnish, so one prep job becomes a sequence of labeled steps rather than one long clip. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events, objects, and per-frame instance masks where the spec calls for them, stored at the depth resolution of 848×480 so they line up with the depth stream.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. The official sources describe three of these in their own terms. The RLDS repository describes an ecosystem of tools to store, retrieve and manipulate episodic data, including for learning from demonstrations and imitation learning. The LeRobot repository describes its dataset format as synchronized MP4 videos or images for vision plus Parquet files for state and action data. The WebDataset repository describes tar files in which files that belong together share the same basename and make up a training sample. In the HDF5 layout the site documents, hand pose sits at annotations/hand_joints and action segments at annotations/action_segments.

The site does not list labels for the quality of food prep. There is no stated label for portion size, plating quality, freshness, temperature or whether a dish is finished correctly. The verb and noun on each action segment is the vocabulary the skill spec defines with you, and anything beyond it is a scoping conversation, not a delivered field.

The GR00T N1 paper structures its training corpora as a data pyramid. In its words: "large quantities of web data and human videos lay the base of the pyramid; synthetic data generated with physics simulations and/or augmented by off-the-shelf neural models form the middle layer, and real-world data collected on the physical robot hardware complete the top." In that framing, first-person human video of food prep would belong to the human-video base. Human capture does not replace robot data, and this page does not claim that it does.

The same paper says of the human video datasets it used: "These do not include explicit action labels but contain extensive sequences of human-object interactions." That is a statement about those datasets, not about this one. A Firsthand episode carries labeled action segments, 3D hand pose and contact events, so the structure of the step is explicit in the data. No model named on this page was trained on Firsthand data, and this page does not suggest otherwise.

Cold prep is a useful family to specify because it is made of small, ordered, close-contact steps: tear, add, dress, toss, plate, garnish. The catalog rates salad assembly Medium and plating Easy. This page does not claim that any robot prepares salads or sushi, that any model improves from this data, or any result.

Public corpora are captured for someone else’s task distribution. A custom collection lets you aim at the prep step your model is missing instead of hoping a public release contains it. This page does not claim that any particular public corpus lacks food prep. It makes the narrower point that a custom spec lets you name what you need and have it captured, labeled and checked to terms you agreed.

The catalog places both rows in an environment named a Working Kitchen Set and lists "food props" among its examples. That wording describes a set rather than a restaurant kitchen, but the site does not say in so many words whether the work is staged or recorded in a live kitchen, so this page does not either. It also does not state whose premises the rows are recorded on, how a kitchen is arranged, a kitchen type, site counts, or regions or cities for these rows.

Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. An undocumented bystander is also a machine-checked reject criterion. The anonymization record is delivered with the data.

If your program needs terms about premises, access or who may be in frame, raise them when you scope it. On food handling, the site states nothing about food safety, hygiene, allergens, health codes or certification for these rows, and this page makes no such claim. None of this is legal advice, and the executed agreement controls.

From signed spec to first validated batch, the site states a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. The site does not state which of those applies to the food prep rows, so that is settled in your quote. Speed comes from a warm pipeline, not from cutting consent, review or anonymization. After the first batch, on-demand top-ups run against the same accept criteria, and the site does not state a turnaround for those later batches.

Validation runs against signed accept criteria written into the spec before capture. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec. A reject log ships with each batch, only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at the provider’s cost.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task family, so those are set in your quote.

Describe the prep steps, the setting and the failure or variation cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first: a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales. A published sample recording is on the sample data outputs page if you want to see the streams before you write a spec.

Three choices matter most for this family. Name the step and the item rather than a broad activity, using catalog wording such as "Tear greens, add toppings, dress, and toss a salad" or "Plate a portion and add garnish," because that is what lets each step become one action segment. If you need sushi, write each step you care about as its own verb and noun, since the catalog gives no sushi wording to borrow. And write the reject rule and the acceptance test first, so the batch is judged against terms you agreed.

Where this site does not state something, this page says "Not stated" rather than filling the gap. The final tables separate what is delivered from what is not stated. There is no sushi row, rolling row for rice or fillings, or cold station row, no label for portion, plating quality, freshness or temperature, no food-safety, hygiene or certification position, and no ingredient list. Counts of episodes, hours, sites and contributors are not fixed numbers on this site. They are set in your skill spec.

The tradeoff to plan for is the embodiment gap between a human hand and a robot end-effector, which retargeting or pairing with robot data has to close. What you get is human first-person demonstrations in which the hands, the food and the step structure are explicit.