Egocentric Baking and Dough Handling Video Data

Egocentric baking and dough handling video data is first-person video of a person kneading dough, rolling it flat, measuring ingredients and mixing a batter, recorded from the contributor’s own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see at a counter, so the frames show the dough under the hands, the dough being rolled flat and the batter being mixed, rather than a distant view of someone in a kitchen. On this site it is a custom collection: you name the task, the setting and the capture tier, and capture is run to a written skill spec.

The data is human capture. A paid, consented contributor wears the rig and does the task. It is not robot or teleoperation data, and it is not a fixed corpus you download. This page covers a different subject from egocentric cooking video data, which spans two-handed food preparation as a whole, from egocentric cutting and food prep data, which centres on knife work, from egocentric container opening data, which centres on jars and packaging, and from egocentric deformable object data, which covers soft shape-changing items in general. The subject here is the dough and the batter: kneading, rolling flat, measuring and mixing.

People reach this topic through close cousins: baking robot training data, a dough handling video dataset, egocentric kneading and shaping data, or custom baking data collection. They describe one request from different angles, and the work is the same: write down the task, collect it on demand, and receive validated episodes. A note on wording: the catalog rows say knead, roll flat, measure and mix, and no row uses the word shaping, so this page claims no shaping task beyond rolling dough flat. Nothing here states that the video carries a dough-type, dough-state, recipe, oven or temperature label, or a force or tactile reading.

A published robot foundation model paper, GR00T N1, linked in the sources below, describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of baking and dough handling sits in that base layer. Human capture does not replace robot data, and this page does not claim that it does.

Dough is a hard case for reasons the site already states. The egocentric kitchen video dataset page lists "Bimanual dough handling" among its skills and describes it as deformable-object manipulation and two-handed coordination. The cooking page adds that dough and produce change shape as they are handled and do not track well with masks propagated frame to frame, so they ship per-frame instance masks. Order matters as well. The catalog describes one row as kneading dough and rolling it flat and another as measuring ingredients and mixing a batter, so a clip that shows a bowl somewhere in frame does not carry what a policy needs. The hands, the dough and the work surface have to be present, and the episode has to say where one step ends and the next begins.

The same kitchen dataset page describes domestic kitchens as "the densest manipulation environment we capture, and the one with the most contact events per minute." That sentence is about kitchens in general, and the site does not say how many of those contact events come from dough. A published model write-up, GEN-0, also linked in the sources, puts it this way: "data quality and diversity matters more than sheer volume." For baking that points at the spread of tasks, settings and people. A custom collection lets you aim at the step your model is missing, such as kneading or mixing, rather than hoping a broad collection happens to contain it.

A published kitchen video dataset, EPIC-KITCHENS-100, is described in its paper as 100 hours of unscripted activity in 45 environments recorded with head-mounted cameras. It is a useful reference for what first-person kitchen video looks like in the public literature. This page does not claim that it lacks dough handling, and it does not state what share of it, if any, shows baking.

The first table below reproduces five rows of the site’s commercial task catalog for the Working Kitchen Set (food prep), the setting described with the examples stovetop, counter, sink, dishwasher, utensils and food props. Two rows name dough or batter directly. "Knead + roll dough" reads "Knead dough and roll it flat." and "Measure + mix" reads "Measure ingredients and mix a batter." Both are medium rows, captured with a multiview rig and handheld grippers, and both are available on request. The other three are neighbouring kitchen rows from the same set: "Crack + whisk eggs", "Slice + spread", which slices bread and spreads a condiment, and "Stir + flip on stovetop".

The catalog does not list oven work, proofing, portioning dough, shaping loaves or pastries, loading or unloading trays or ovens, or decorating and icing as tasks, so this page does not describe them as available. If your model needs one of them, it is settled in a Spec Pilot rather than assumed. The site does not name a dough type, a recipe, a batter, an ingredient quantity, a tool, a bowl size or an oven for any row.

Two other catalog environments mention baking-adjacent places only as examples. The Restaurant & Commercial Kitchen environment lists "artisan bakery, patisserie" among its examples, and the fast food environment lists a pizza counter. None of the rows under them names baking: the commercial kitchen rows cover food prep, line cooking, knife work, portioning and packaging, dishwashing, cleaning, trash removal and line setup. This page therefore makes no claim that episodes are recorded in a bakery, a patisserie or a pizza counter.

The site names no bakery, brand or property for these rows, and it lists capture at your own facility as not offered. The sample data outputs page also includes a ten-second US home-baking recording, covered below, but the site does not say which baking step it shows. The environment names are the catalog’s own, and which of them applies to your program is set in the spec.

Each attempt is delivered as one episode, and a baking task needs a decision up front about what one attempt is: one knead, one dough rolled flat, one measured and mixed batter, or a longer run that strings several steps together. The site does not set that length, so the skill spec does. Inside the episode, the synchronized streams run from start to end, and action segments divide it into labeled spans.

A dough episode might open with the dough on the surface and close when it has been rolled flat, and a batter episode might run from measuring the first ingredient to the end of mixing, as the catalog rows describe. These are illustrations of how a spec can cut an episode, not stated durations. Whether kneading and rolling flat are one episode or two, and whether measuring is part of the mixing episode or its own, are spec decisions.

The setting behind these rows is the Working Kitchen Set, captured with a multiview rig and handheld grippers according to the catalog. How many tasks, people, kitchens and conditions you want covered is set in the spec, along with whether you want a few long runs or many short ones.

A baking collection uses the same capture stack as every Firsthand program. The instrumented tier is a head-mounted stereo rig that records first-person RGB, with a metric depth stream recovered from the calibrated stereo rig rather than from a standalone depth sensor, and 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors. The wider stack is ten streams: a 4K head camera, two wrist cameras, depth, IMU, 3D hand and body pose, instance segmentation and action labels, hardware-aligned under 2 ms. The two wrist cameras give a closer view of the hands than the head camera alone, which matters where both hands work the same piece of dough.

Every stream is hardware-triggered onto one PTP clock domain, so depth, motion and hand pose line up with the same video frame rather than being approximated afterwards. The rig is calibrated twice per session, before capture starts and after it ends, and an episode is accepted only if RMS reprojection stays under 0.28 px. The catalog lists the same capture setup for all five rows in the first table, a multiview rig with handheld grippers, and the site does not explain the grippers further. Which tier applies to your program is set in the skill spec.

The sample data outputs page shows the delivered shape of this stack on a baking recording: a ten-second home-baking clip filmed in the US, with the IMU rows for the same ten seconds. Faces are blurred and the web clip is downscaled to 1080 × 1080, while the delivered files are the full 4608 × 4608 frame. The site does not say which baking step that clip shows, so treat it as a reference for the file format, not a claim about a task. The site states no camera count or frame rate specific to dough tasks. Force and tactile readings are not among the streams, nothing measures how firm a dough is, and no robot or teleoperation data is captured.

Inside each episode, action segments divide the recording into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For this task family a segment can be one step as written in the skill spec, so a batter becomes a run of labeled steps such as measure and mix rather than one long clip. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets an object, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.

Hand pose is delivered for both hands with a per-joint visibility flag for occluded fingers. That is the label that carries how the hands press, fold, roll and stir. The site describes contact events as the moment the hand meets an item, and it does not say whether dough, a bowl or the work surface counts as the item, so the spec is the place to settle what a contact event means for your task.

The site does not list labels for the dough or the batter itself. There is no stated label for dough type, dough state or consistency, proofing, ingredient or quantity, recipe step, oven setting or temperature, or a task-completion score, and no contact force reading. Catalog descriptions such as kneading dough and mixing a batter describe the task the contributor performs, not a delivered measurement.

Several limits the site already states apply directly to dough and batter capture, and several it leaves open. Each stated limit below appears elsewhere on the site rather than being introduced here.

Hand occlusion is the first. Fingers disappear behind objects in manipulation, which is why 3D hand pose carries a per-joint visibility flag, and dough is worked under and between the hands. Masks are the second. The cooking page says dough and produce change shape and do not track well with masks propagated frame to frame, so per-frame instance masks are shipped for them. Depth is the third. The site says depth degrades on wet and reflective surfaces and handles that with per-frame confidence maps, and the kitchen dataset page adds that steam events are labelled as an episode-level flag because they wreck depth for 2 to 6 seconds at a time. The site does not say how depth behaves on dough, on a bowl of batter or on a particular work surface, and it does not say that any dough row involves steam.

The fourth is the set of machine-checked reject criteria. An episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, calibration drift beyond tolerance, the manipulated object out of frame for more than 15 percent of the episode, more than 35 percent invalid depth pixels in the manipulation region, an undocumented bystander, or a mismatch against the named skill spec. A human reviewer then checks each episode against the task spec. Kneading keeps the hands close to the work, so what counts as the manipulated object, the dough, the bowl or the batter, is worth settling in the spec.

Also not stated: a mask accuracy figure for dough edges, and any description of how a kneaded dough, a rolled sheet or a mixed batter is judged finished. If your policy depends on either, say so at the start.

Validation runs against signed accept criteria written into the spec before capture. Each batch goes through per-batch QA and a reject log ships with it. Only footage that clears the criteria is billed as a validated hour, and rejected items are recaptured at our cost. Many teams start with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales.

Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. The consent chain has six checkpoints, from before capture to after delivery, and a break at any one discards the footage rather than shipping it at a lower grade. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. The anonymization record is delivered with the data.

On safety and qualification, the site states no food-safety or hygiene protocol, no allergen guidance and no baking or food-handling certification for any dough row, and this page does not describe one. If your program needs terms about who performs the work, the ingredients used or the conditions under which it is done, raise them when you scope it. None of this is legal advice, and the executed agreement controls.

From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. Speed comes from a warm pipeline, not from cutting consent, review or anonymization. The site does not say whether the Working Kitchen Set counts as deep or thin coverage, so ask which timeline applies to the setting you want when you request the quote. After the first batch, on-demand top-ups run against the same accept criteria, so you can add more tasks, people or conditions without rewriting the spec. The site does not state a turnaround for those later batches, so plan them in your quote.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader. Collections are billed per validated hour.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for this task family, so those are set in your quote.

Where this site does not state something, this page says "not stated" rather than filling the gap. The table below separates what is delivered from what is not stated. There is no listed label for dough type, dough state, ingredient, recipe or oven, no force or tactile stream, no temperature data, and no bakery or commercial-kitchen claim beyond the catalog rows and examples quoted above. Nothing here claims a training outcome or a model improvement.

The tradeoff to plan for is the embodiment gap between a human pair of hands working dough and a robot platform doing the same, which retargeting or pairing with robot data has to close. Counts of contributors, hours and tasks are set in your skill spec.

Describe the steps, the setting and the variation or failure cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first. The scoping checklist below maps the parts of a skill spec onto a baking request. A published home-baking sample recording is on the sample data outputs page if you want to see the streams before you write a spec.

Three choices matter most for this family. Name the step and the item rather than a broad activity, because that is what lets each step become one action segment. Name the setting and the capture tier, since the catalog lists the Working Kitchen Set rows with a multiview rig and handheld grippers. And name where the episode starts and stops: one knead, one rolled sheet, one mixed batter or a longer run, since the site does not set an episode length.