Stereo Fast Food Fryer and Grill Data

It is first-person video of work on the line of a quick-service kitchen: dropping baskets into the fryer, flipping items on the grill, timing the cook and draining product, then assembling items in order and placing them on trays. It is recorded from the worker’s own point of view with two calibrated cameras, with hand pose and sensor streams alongside. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a head-mounted phone or a multiview rig as their capture setup, so a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data, so using it for a specific robot still needs your own retargeting step. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Three, all in the Fast Food & Quick Service Kitchen environment of the Food & Beverage sector, and all listed in lib/commercial-catalog-data.ts. "Fryer and grill operation" is described as "Drop baskets into the fryer, flip items on the grill, time cooking and drain product." and rated Medium. "Order assembly" is described as "Assemble burgers, wraps and sides from the line in order, wrap and place them on trays." and rated Easy. "Counter service and packaging" is described as "Take orders at the counter, bag or box items, add condiments and hand them to customers." and rated Easy.

All three are listed as Available on request. The catalog does not define that status further, and the site says nothing about delivered hours or delivered stereo footage for any row, so this page claims none. The environment examples read "Burger chain, fried chicken, pizza counter, sandwich shop, fast-casual kitchen"; these are kinds of operation, not named restaurants, and this page names no chain or equipment brand. The first table below reproduces the rows as the catalog states them.

The egocentric fast food and quick-service page covers the same three rows as first-person catalog capture, with ten streams synced under 2 ms, and stereo and MCAP are not its subject. The stereo commercial kitchen line cooking page covers two Restaurant & Commercial Kitchen rows, Culinary Chef Work / Knife Work (Hard) and a vegetable chopping and portioning row (Medium), both Active collection; neither is a fryer, grill or quick-service row. The egocentric stereo MCAP page is the hub across tasks.

This page is the stereo and MCAP treatment of the three quick-service rows only, with its own stream map and failure modes. No egocentric grill and barbecue page exists on the site, so none is compared. The second table below sets the pages side by side.

The catalog lists "Phone (head-mounted) / multiview rig" for all three rows. The site does not state a head-mounted stereo setup for them. Stereo applies through the site general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot.

A multiview rig adds synchronized third-person angles, and wrist cameras give a closer view of the hands, when the spec asks for them. Whether a stereo pair is worn, mounted or combined with a multiview rig on these rows is Not stated. The site also states that Firsthand does not capture at a customer’s own facility, so which kitchens a program uses is agreed in the spec.

Line work is close and cluttered. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so a basket, a grill surface, a utensil, a wrap and the hands around them share a frame. The site describes a head camera as sitting close to the work, as close as a robot onboard camera would, and gives no depth accuracy figure for these rows.

A stereo pair adds depth, not a verdict. The site states 1080p per eye for dataset 06, and whether that suffices to see a flipped item or a basket handle at working distance is Not stated, so a spec should check it on a Spec Pilot.

The row names four steps: dropping baskets into the fryer, flipping items on the grill, timing the cook and draining product. It is the only Medium row of the three. The site states no fryer or grill type, no temperature, no cook time and no food-safety or hygiene procedure for it, and this page adds none. Whether a timer is visible in the frame is Not stated.

Because the steps chain, action segments matter: a spec that wants drop, cook, lift and drain separated has to name them, and the site does not set an episode length. Whether an episode covers one basket, one batch or a whole order is a choice for the spec.

Order assembly and counter service are the two Easy rows. The first is described as assembling burgers, wraps and sides from the line in order and wrapping them onto trays; the second as taking orders at the counter, bagging or boxing items, adding condiments and handing them over. Both can sit in the same episode as fryer and grill work if the spec says so, since the three rows share one environment and one capture setup.

The site states no menu, recipe, order volume or ticket time for these rows. Contact events and action segments can mark when an item is picked, wrapped, bagged or handed over where the spec asks for them, aligned to video and depth.

Baskets, utensils, wraps, bags, trays and the fingers themselves can cover the item being worked, and the head camera sits close to the work. The site states that fingers disappear behind panels and tools, which is why each hand joint carries a visibility flag, and the fast-food page notes that egocentric video puts hands entering the frame from below, occlusion during grasp and motion blur during reach into the training distribution.

The site gives no minimum object size and no depth figure at working distance for these rows. How often a finger is hidden during a basket drop or a grill flip is Not stated, so a spec can ask for wrist views or a second angle and check it on a Spec Pilot.

The site states that depth degrades on wet and reflective surfaces and ships per-frame confidence maps so a buyer can mask rather than guess. Its food service page states that wet, reflective and hot surfaces get depth-quality flags on affected episodes, which is a program-level statement and not a row-level claim for the fryer row. The egocentric cooking page states an episode-level steam flag; whether it applies to these rows is Not stated.

How a stereo pair behaves on stainless steel, a grill plate, fryer steam or a glossy wrap is Not stated for these rows, so it is a question for a Spec Pilot. The site gives no lighting setup for them either.

Only the program-level statement above: hot surfaces get depth-quality flags on affected episodes. The catalog rows state no safety rule, and the site states no food-safety or hygiene procedure, certification, oil or grill temperature, cook time, ticket time or order volume for them. This page states none either, and any requirement belongs in the written spec.

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For these rows the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Hand and utensil occlusion is the first risk. A basket, a utensil, a wrap, a tray or a finger covers the item, which is why each hand joint carries a visibility flag. An episode is rejected when the manipulated object is out of frame for more than 15 percent of it. For quick-service work the spec decides what that object is: the basket, the item on the grill, the wrap or the tray.

Invalid depth near the work is the second. An episode is rejected when more than 35 percent of depth pixels in the manipulation region are invalid. Calibration is measured at the start and end of every session under a 0.28 px RMS gate, and an episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, for an undocumented bystander or for a mismatch against the named skill spec. A reject log ships with every batch.

Whether an item was cooked correctly is Not stated: the site lists no doneness, success, defect or quality label for these rows. The fifth table below lists the failure modes beside what the site states.

For every delivered episode the site states that faces, plates and screens are blurred irreversibly, that a reviewer confirms the result before anything ships, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data, and an undocumented bystander is a stated reject criterion. In a counter setting, customers are the bystanders this applies to.

The site does not state a rule specific to a restaurant, to branded packaging or to menu boards, so this page makes no such claim. Settle it in the spec.

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. The site names no fryer, grill or quick-service dataset among them, so this work is custom collection, not an off-the-shelf purchase.

It is custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The site does not say which speed applies to these rows, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Describe the steps and where each starts and ends: dropping a basket, flipping an item, draining product, assembling an order, handing it over. Say whether an episode covers one basket, one batch or a whole order, and name the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The sixth table below shows what each spec element decides.