Egocentric Bartending Video Data

Egocentric bartending data is real-world video recorded from the point of view of a person who is making drinks at a bar. The camera sits on the head, so the footage shows what the person sees: the hands, the vessel, the garnish and the counter. Alongside the video, each episode carries depth, 3D hand pose, contact events and frame-accurate action segments, all aligned on one hardware clock.

Teams search for it under several names, including bartending robot training data, a cocktail prep video dataset and egocentric bar task data. They all describe first-person demonstrations of bar work, collected to a written spec. This page covers only what the Firsthand site states about it and marks everything else as Not stated.

The site states bar work in its commercial task catalog, in two environments. Hospitality Service Set has a Prepare and garnish drink row, "Build and garnish a bar drink", and a Pour and serve beverage row, "Pour a drink and serve to a place". Cafe, Bar and Beverage Service has a Professional Barista Workflows row, "Grind beans, dose and tamp, pull espresso shots, steam milk and pour drinks at the bar".

Anything outside those rows is Not stated. That includes shaking, stirring, muddling or straining a drink, measuring with a jigger, cutting fruit for a garnish and taking an order, so none is claimed here. If your policy needs one of them, you write it into the spec and we scope it, because the catalog rows are a starting vocabulary, not a fixed menu.

All three rows are rated Medium and listed as Available on request. The two Hospitality Service Set rows are recorded with a multiview rig and handheld grippers. The barista row is recorded with a head-mounted phone or a multiview rig.

Available on request means the work is scoped and recruited to spec when a buyer asks for it. None of these rows is listed as Active collection, so the site does not say that a bar capture stream is running today. A first batch is therefore quoted against your spec rather than drawn from a stream that already exists.

The site has no environment called a bar station or a bartending set. Its Cafe, Bar and Beverage Service environment gives a coffee shop, a juice bar, a cocktail bar and a hotel bar as examples. Its Hospitality Service Set describes a table, a counter or bar and a housekeeping or room area.

The two environments differ in capture mode. The barista row uses the head-mounted phone mode, which the site describes as a first-person view of real work on site, and it also lists the multiview rig. The Hospitality Service Set rows use the multiview rig with handheld grippers, which the site describes as a staged working set. So the site describes a real workplace for the barista row, and a staged set for the two bar-drink rows.

The site names no bar, venue or company, and does not state the number of sites, the regions or the venue type. It does not state capture at a customer own facility either. Environment, access and schedule are agreed in the written spec.

A head-mounted phone records the first-person view of the job, and a multiview rig adds synchronized third-person angles when the spec asks for them. On the staged set, the operator performs the task with handheld grippers, so the demonstration maps onto a robot end effector. A wrist-mounted camera is available alongside the head camera for close-range detail. Camera count, resolution, frame rate and bar layout are Not stated, so raise any of them in the spec.

Each episode follows the Firsthand episode schema: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. For bar work, hand pose tracks the fingers as they hold a vessel and place a garnish, contact events mark when a hand meets or releases an object, and action segments mark each step of the build.

Object labels use the terms your spec sets. Drink or recipe counts, glassware and bar-tool types, ingredient lists and pour volumes are Not stated. Pour accuracy, task durations, cycle times and the number of episodes are Not stated too. Success or quality labels, such as whether a drink was built correctly, are Not stated, as are certifications and robot results. The site makes no safety-compliance claim for this work.

The catalog rates these rows Medium, so the site does not call them Hard, and this page makes no claim about robot results. The site gives no technical analysis of bar work. What the row wording implies is a sequence: a build is several steps in order, a pour moves a vessel to a place, and a garnish is a small placement at the end of a longer job.

The viewpoint matters too. The site states that a robot or humanoid perceives from a head- or wrist-mounted camera, not a static overhead shot, and that egocentric capture matches that geometry.

A good spec treats each named job as its own task. Building a drink, garnishing it, pouring one and serving it each get a start state, an end state and a set of accept criteria. That keeps action segments consistent across contributors and lets you ask for more of one job without re-collecting the others.

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. Bartending episodes use the same schema as every other Firsthand collection, so a loader that reads another Firsthand dataset reads this one without changes. The format table below sets each format against what its official documentation states.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch. Rejected items are recaptured at our cost, and billing is per validated hour.

Not in standard capture. Force or tactile readings are Not stated for the bar rows. The site separately offers a tactile glove tier as a pilot-ready option, in which glove signals are time-synced to a head-mounted stereo camera by hardware timestamps and an IMU. The site does not state that the tier has been used on bar work, so if your policy needs a contact signal, ask for it in the spec and agree the fit with us first.

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. A vision model runs on every frame at ingest, flagging faces, heads, licence plates and any legible screen or document. Those regions are irreversibly blurred, and a human reviewer confirms the result before footage ships.

If the reviewer is unsure a region is fully covered, the episode is held or discarded, never downgraded and shipped. A bar can put customers, colleagues and point-of-sale screens in frame, which is why that pass matters here. The site does not state that bottle labels or brand marks are blurred, so raise any such need before the spec is signed. It states no age-verification, licensing or other legal terms for this work, and no in-home capture.

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that up front. The site does not state how deep coverage is for the bar rows, and they are Available on request, so plan on a quote rather than assuming the 48 hour figure.

The site also states program milestones. A pilot reaches its first delivery in two to three weeks, and a standing program reaches full rate in four to six weeks. Those assume capture coverage is already in place. On-demand top-ups run against the same spec and accept criteria after each training run. A batch size and a later-batch turnaround are Not stated, so both are quoted against your spec.

You describe the bar jobs, the environment and the sequence your policy needs. We turn that into a written spec covering tasks, environments, capture tier, annotation depth and accept criteria, then quote reach, timeline and price before any capture begins. This is the custom egocentric data collection route that every Firsthand task family follows.

A Spec Pilot, a small and fast first batch, can prove the spec and accept criteria before you scale. Every collection ships under a perpetual, buyer-owned commercial license, with a reject log and signed accept criteria on every batch.