Stereo Barista and Espresso Data

It is real-world, first-person video of a person working a coffee bar, recorded with two calibrated cameras so that depth can be recovered, together with 3D hand pose and the other streams the site already lists. The person grinds beans, doses and tamps, pulls espresso shots, steams milk and pours drinks at the bar. The buyer sets the task and the accept criteria; Firsthand handles contributors, capture, QA, consent and delivery.

The data is human capture by paid, consented contributors, not robot or teleoperation data. This page is about barista robot training data in the sense of what a policy can learn from watching skilled bar work, and it makes no claim that any robot has been trained on it.

One. In the Cafe, Bar & Beverage Service environment of the Food & Beverage sector the catalog lists "Professional Barista Workflows": "Grind beans, dose and tamp, pull espresso shots, steam milk and pour drinks at the bar." It is rated Medium, its status is Available on request, and the capture setup it lists is Phone (head-mounted) / multiview rig. The environment examples are a coffee shop, a juice bar, a cocktail bar and a hotel bar. The first table below reproduces the row exactly.

Available on request means the work is scoped and recruited to spec when a buyer asks for it. This page therefore claims no completed collection, no delivered hours and no delivered stereo footage for this row. Three other rows are context only: "Prepare + garnish drink" and "Pour + serve beverage" in the Hospitality Service Set, and "Pour-over coffee" in the Working Kitchen Set. They are listed in the same table and are not the subject of this page.

Not stated. The catalog lists a head-mounted phone or a multiview rig for the barista row and no stereo pair. Stereo applies here only through the general stereo offer: the site describes a stereo head rig and a hard-environments stack, and a stereo pair for a barista order is a choice made in the skill spec and confirmed on a Spec Pilot.

This is why the page is written as a stereo and container page for the one row rather than as a description of existing stereo coffee bar footage. If you need stereo, say so in the brief; if you are happy with the catalog setup, the egocentric barista page describes that non-stereo route.

This page is stereo capture of espresso and milk-drink preparation at a coffee bar. The egocentric barista and espresso page covers the same row without stereo. The egocentric beverage and coffee prep page treats drink making as a family of tasks, with barista work as one row among them, and the egocentric bartending page covers bar and cocktail work. None of the three is stereo.

The two stereo kitchen pages are a different setting, not a different capture type. The stereo commercial kitchen line cooking page covers knife work and line cooking in the Restaurant & Commercial Kitchen environment, and the stereo dishwashing and kitchen sanitation page covers the dish pit and deep cleaning in the same environment. The second table below puts all of them side by side with their status as the site lists it.

Grinding beans is the first step: a short, repeated hand motion at the start of a longer chain, where the data needs to show where it ends. Dosing and tamping follow, placing the ground coffee and pressing it down, small fine-motor steps where the fingers and the pressing hand matter more than the wide scene. Pulling the espresso shot comes next, and the key signal is the order of the steps and the moment a hand starts and stops each one.

Steaming milk and pouring drinks close the sequence. Milk moves from one vessel to another and the drink is poured at the bar, so a spec decides whether the pour is a step of its own or the end of the steam. The site states no drink list, recipe or vessel size for the row, so which drinks are made and in what vessels is agreed in the spec rather than assumed. Action segments carry the start and end of each step as the spec defines them.

The site describes two stereo configurations. The stereo head rig, dataset 06, is a dual fisheye pair at 1080p per eye on a 62 mm baseline, with per-unit calibration, a 9-axis IMU and hand-tracking output, shipped as MCAP. The hard-environments stack is a 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, depth at 848×480 and 30 fps and a 200 Hz 6-DoF IMU. Which of these a barista order uses is Not stated and is agreed in the skill spec.

Whichever rig is chosen, calibration is measured every session and shipped with the episode, with a 0.28 px RMS reprojection gate. Hand pose is 21 joints per hand with a per-joint visibility flag, and the language data is one task instruction per episode. Camera count, resolution and frame rate for barista work specifically are Not stated, and so are the cafe, machine or grinder a collection runs in or on.

Hand occlusion is the first case. The portafilter, the pitcher and the equipment at the bar can hide the fingers that hold them. The per-joint visibility flag on the 21-joint hand pose marks joints that are not visible instead of guessing them, contact events mark when a hand meets an object, and a spec can ask for wrist or multiview angles so a grip the head camera loses is still recorded.

Depth is the second. The site states that depth degrades on wet and reflective surfaces and that per-frame confidence maps mask low-confidence regions, so a consumer of the data can tell where depth should not be trusted. How depth behaves around steam, liquid, steel or glass at a particular bar is Not stated, so a Spec Pilot checks it on real footage. Lighting at a bar is the third case: the site states no standard lighting values for barista work, so any lighting conditions are fixed in the skill spec for the bar in question.

Every batch is validated against signed accept criteria before it ships, with a reject log. The site states that an episode is rejected when the manipulated object is out of frame for more than 15 percent of it, when more than 35 percent of depth pixels in the manipulation region are invalid, or when an undocumented bystander appears. Episodes that fail are rejected and logged per batch, not delivered with a caveat.

Billing is per validated hour, and rejected items are recaptured at our cost. The site does not state an accuracy figure for step labels, so what counts as a pass for a step boundary is set in the accept criteria.

Firsthand offers MCAP delivery. MCAP is an open-source container format with its own official site at mcap.dev, a specification at mcap.dev/spec and a repository at github.com/foxglove/mcap; read those for what the container is and how it stores records, timestamps and attachments. This page describes the container only from those sources and does not state anything more about it.

The streams the site already states for a stereo capture are the stereo pair, depth with confidence maps, 21-joint hand pose, IMU, contact events, action segments and calibration. Topic names, message schemas, encoding and rates are Not stated on the site and are agreed in the skill spec. Only dataset 06 is stated as shipped as MCAP; for any custom order, MCAP can be the only delivery or sit beside RLDS, LeRobot, WebDataset, HDF5 or zarr.

Drink recipes, vessel sizes, liquid volumes, fill-level labels, dose weights, extraction times, milk temperatures, drink counts, task times, error rates, force or tactile data, a task-completion label and any food-safety or hygiene certification are not stated on the site as standard fields. No coffee brand, machine or grinder model, cafe or customer is named for the row.

If your policy needs any of these, they are written into the skill spec as annotation depth and covered in the quote before capture begins. This page gives no equipment-safety, food-safety or other operating instructions.

The site’s off-the-shelf catalog lists about 324,800 hours across 18 datasets, including about 64,800 hours of stereo video in 9 datasets. None of them names barista or espresso work. One stereo set, S2 (~5,000 hours, Cambodia, 2, 4 and 6 camera rigs, stereo RGB, task labels and hand keypoints), lists cafés among its environments and "food and coffee prep" among its tasks. Whether S2 holds any barista step is Not stated, so it is a way to look at the format and rig, not coverage of this row.

Stereo barista collection to a spec is therefore custom collection: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The row is Available on request, so the scoped-to-spec timeline applies rather than the ready-now case, and the quote states it up front.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships, with the anonymization record delivered alongside. A coffee bar can put customers in frame, which is why that pass matters here. Text on cups and packaging is not listed as a blur category, so this page does not claim it is blurred; raise it in the spec if it matters.

The buyer owns the dataset under a perpetual, irrevocable, non-exclusive-by-default licence, with exclusivity available. Consent is recorded per session.

Describe the steps, the drinks, the equipment and the bar you expect to see. Say whether one episode is a single drink, a single step or a run of drinks, what counts as a reject, and whether you want a stereo pair, wrist views or a multiview rig. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The last table below shows what each spec element decides.