Meal plating and serving

Egocentric Meal Plating and Serving Video Data

Portion the dish, set it on the plate, finish it, load the tray and carry it to the table: plating and serving is a chain of small, precise two-handed steps that ends in front of a guest. We collect custom egocentric video of plating, portioning and serving on demand, to a written skill spec, with synchronized streams, 3D hand pose, contact events, action segments, anonymization and a buyer-owned license.
Tasks
Plating, portioning and serving rows, as the catalog lists them
Streams
Ten, synced under 2 ms
First batch
Median 48 h in deep-coverage settings
License
Perpetual, buyer-owned

Requirements

What does plating and serving need that a generic cooking clip does not?

01

Order, placement and finish

The catalog describes plating as a sequence: portion hot dishes onto plates in sequence, wipe rims, load plates onto service trays; plate a portion and add garnish. The value is in the order of the steps, where each item lands on the plate and how each step ends.

02

Fine hand work in view

Placing, spooning and carrying are done with fingertips and a steady wrist. First-person capture keeps both hands in frame, and the two wrist cameras add a closer view. 3D hand pose carries a per-joint visibility flag, so a finger hidden behind a plate or a tray is flagged rather than guessed.

Definition

What is egocentric meal plating and serving video data?

It is first-person video of a person plating, portioning or serving food and drinks, recorded from a head-mounted camera, with two wrist cameras for a closer view of the hands. The contributor is a paid, consented person following a written skill spec, not an actor and not a robot.

The point of the data is what a hand does at the end of the chain: placing a portion, finishing a plate, setting a tray down in front of someone. It is labeled so each step can be sliced out. Where the site states nothing, a table says "Not stated" and the answer lives in the spec or the quote.

Task coverage

Which plating and serving tasks can be captured?

The task catalog names eleven rows that are explicitly about plating, portioning or serving. Three are plating rows: "Banquet plating", "Plate + garnish" and "Cooking / Line Cooking", whose description includes plating and service. Four are portioning rows: "Kitchen Meal Portioning & Packaging", "Portion and package meals using a kitchen scale and food-grade containers", "Chop and portion produce using a chef’s knife and cutting board" and "Food Prep".

Four are serving rows: "Pour + serve beverage", "Tray assembly + delivery", "Counter service and packaging" and "Order assembly". Statuses vary: Ready to deploy, Active collection or Available on request, and the catalog does not define any of them further, so ask which applies to your row.

The second table lists neighbouring rows, such as setting a table, that are not plating or serving rows themselves.

Settings and rigs

Where is plating and serving data captured, and on what rig?

The catalog ties these rows to four environments. Catering & Event Kitchen lists examples such as a hotel banquet kitchen, airline catering and a hospital canteen. Restaurant & Commercial Kitchen lists a sit-down restaurant kitchen, a café and an artisan bakery. Fast Food & Quick Service Kitchen is the third. Hospitality Service Set is described as a table, a counter or bar, and a housekeeping or room area.

The rig differs by row. Kitchen rows list a head-mounted phone or multiview rig, and the Working Kitchen Set and Hospitality Service Set rows list a multiview rig plus handheld grippers. The site does not state capture at a customer’s own facility.

Episode contents

What does each plating and serving episode contain?

An episode is one contributor performing one task from the skill spec, recorded with ten hardware-synchronized streams that line up within 2 ms. The site documents a canonical episode schema, and every delivery format is a view of it.

Hand pose is stored as 21 joints per hand in metres in the head frame, with a visibility flag for each joint. Contact events record when a hand meets an object. The site does not say whether a plate, a utensil or a tray counts as the object, so settle it in the spec.

Action segments carry a start, an end, a verb and a noun from a taxonomy the site states as 97 verbs and 512 nouns, human-reviewed. For plating, a segment can be one step as written in the spec. Every episode links to its contributor’s signed release by a consent artefact ID, and each batch ships with a data card.

Labeling

How is plating and serving data labeled?

The unit of labeling is the action segment. A spec that says "plate the meal" gives you one long segment. A spec that names portion, place, garnish and load gives you a sequence you can slice by step.

Every episode is machine-checked, then reviewed by a human against its skill spec, and rejected episodes are recaptured at our cost. The site states no label for dish type, portion weight, presentation quality or task success, and no force or tactile reading, so ask for any such label at the start.

Privacy

How is privacy handled when capturing in kitchens and dining rooms?

Only what the site already states applies. Every contributor signs a written release, and each episode links to it. Faces and screens are detected and blurred irreversibly, a human reviewer confirms that no un-consented person is identifiable, and footage with bystanders who cannot be de-identified is discarded rather than shipped.

The site does not state how guests or customers in a dining room or a service area are handled beyond that general rule, so settle it in the spec before capture. This is not legal advice, and the executed agreement controls.

Delivery

How is plating and serving data delivered, and who owns it?

Data is delivered in RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or as raw streams with metadata, with the episode schema as the canonical layout. QA runs once, against the canonical episode, before any format is cut. The formats table sets the site’s delivery next to each maintainer’s own documentation, linked under Sources.

The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity available as a separate term and the executed order form governing.

Speed

How fast can a custom plating and serving dataset be collected?

The site states one published timeline: from a signed spec to a first validated batch, a median of 48 hours where coverage is already deep, and typically three to four weeks where it is thin because operators have to be staffed. The site does not say which applies to the plating and serving rows and promises nothing faster, so ask when you request the quote.

Collection is on demand, so a focused brief, such as one banquet plate from first portion to loaded tray, is a normal starting point.

Getting started

How do you start a pilot?

Start with the free sample pack to check the loader and label quality, then move to a Spec Pilot: a small, fast first batch against the written spec that proves the fill rate and reject rate on your conditions before you scale.

A good plating brief names the setting, the sequence of steps, how each step starts and ends, what counts as a reject and the privacy rule for the room. The site’s skill-spec guide walks through that structure. Then request a custom dataset.

Capture setup

What is the capture setup for a plating and serving collection?

The capture setup as stated on this site, applied to plating and serving. Counts of contributors and hours are set in the skill spec.
PartAs stated on this site
Head cameraGlobal-shutter 4K at 60 fps with a 120 degree lens
Wrist camerasTwo wrist-mounted cameras at 1080p and 60 fps
DepthDepth sensor at 848×480 and 30 fps, with per-frame confidence maps
IMU6-DoF at 200 Hz
SyncTen streams, hardware-triggered onto one PTP clock domain and aligned under 2 ms
CalibrationRe-solved with a checkerboard target at the start and end of every session; accepted only under the 0.28 px RMS reprojection gate
Catalog capture setupHead-mounted phone or multiview rig for kitchen rows; multiview rig plus handheld grippers for Working Kitchen Set and Hospitality Service Set rows
Dish types, plate and utensil types, portion weightsNot stated

Every collection runs through the same seven-stage end-to-end custom collection pipeline.

Task catalog

Which plating, portioning and serving rows does the catalog name?

The eleven catalog rows that are explicitly about plating, portioning or serving, reproduced as the catalog states them, with named food items generalized.
Task (as listed)Description (as listed)EnvironmentSkill levelStatus
Banquet platingPortion hot dishes onto plates in sequence, wipe rims and load plates onto service trays.Catering & Event KitchenMediumAvailable on request
Plate + garnishPlate a portion and add garnish.Working Kitchen Set (food prep)EasyAvailable on request
Cooking / Line CookingFront and back of house restaurant operations including prep, plating and service.Restaurant & Commercial KitchenMediumActive collection
Kitchen Meal Portioning & PackagingWeigh cooked food into portions, fill containers, seal, label and stack them for service.Restaurant & Commercial KitchenEasyReady to deploy
Portion and package meals using a kitchen scale and food-grade containersWeigh each portion on the scale, fill containers to spec and seal and label them.Restaurant & Commercial KitchenEasyActive collection
Chop and portion produce using a chef’s knife and cutting boardWash, trim and chop produce into even pieces and portion them into containers.Restaurant & Commercial KitchenMediumActive collection
Food PrepBack of house food preparation, staging, portioning and storage.Restaurant & Commercial KitchenMediumActive collection
Pour + serve beveragePour a drink and serve to a place.Hospitality Service Set (table, bar, room)MediumAvailable on request
Tray assembly + deliveryArrange, carry, and set a tray.Hospitality Service Set (table, bar, room)EasyAvailable on request
Counter service and packagingTake orders at the counter, bag or box items, add condiments and hand them to customers.Fast Food & Quick Service KitchenEasyAvailable on request
Order assemblyAssemble items and sides from the line in order, wrap and place them on trays.Fast Food & Quick Service KitchenEasyAvailable on request

Neighbouring rows

Which catalog rows sit next to plating and serving?

Rows that touch plating or serving but are not plating or serving rows, reproduced as the catalog states them. They are listed for context.
Task (as listed)Description (as listed)EnvironmentStatus
Set table serviceLay a full place setting.Hospitality Service Set (table, bar, room)Available on request
Clear + bus tableStack dishes, bus, and wipe.Hospitality Service Set (table, bar, room)Available on request
Prepare + garnish drinkBuild and garnish a bar drink.Hospitality Service Set (table, bar, room)Available on request

Episode fields

What fields does a plating and serving episode carry?

Episode fields as documented in the site’s episode schema, and what each is useful for in a plating task.
FieldAs documentedUse for plating and serving
hand_joints21 joints per hand, in metres, head frameHow each hand places, spoons or carries
hand_visibleBoolean visibility flag per jointWhich fingers are hidden behind a plate or tray
contact_eventsTimestamp, hand, object ID and kindWhen a hand meets a plate, utensil or tray
instance_masksTracked object IDs per frame at 848×480Which object is which across frames
action_segmentsstart_ns, end_ns, verb, nounOne labeled span per plating or serving step
depth_mm and depth_confMetric depth with a per-pixel confidence mapWhere depth can be trusted near the plate
Dish type, portion weight, presentation or success label, force or tactileNot statedNot stated

Delivery formats

Which formats are delivered, and what does each format’s own documentation say?

The delivery formats the site lists. The third column is taken only from the official pages linked under Sources.
FormatDelivered by FirsthandMaintainer documentation
RLDSTFDS-compatible tfrecord shards, one skill line per directoryStores episodic data; a dataset is a tf.data.Dataset of episodes, each containing a dataset of steps
LeRobotParquet plus an mp4 per camera, in the Hugging Face datasets layoutSynchronized MP4 videos or images for vision, plus Parquet files
WebDatasetListed as a delivery formatTar files in which files sharing a basename make up one training sample
HDF5, zarr and Rerun .rrdListed as delivery formatsNot described on this page

Not stated

What does the site not state about plating and serving?

Details a buyer may ask about and where the answer lives.
TopicOn this siteWhere it is settled
Dish and cuisine types, plate, utensil and tray typesNot statedWritten skill spec
Portion weights and presentation standardsNot statedWritten skill spec
Force, tactile or torque dataNot statedNot offered as a stated stream
Presentation or task-success labelsNot statedScoped in the spec
Food-safety or hygiene certificationsNot statedNot offered as a stated claim
Throughput, and which timeline applies to these rowsNot statedQuote
Guests or customers in frame in dining roomsNot stated beyond the general bystander ruleSkill spec
Robot actions or teleoperation dataNot stated; Firsthand captures humans, not robotsYour own retargeting step

FAQ

Egocentric meal plating and serving video data, answered.

What is egocentric meal plating and serving video data?
Custom first-person video of people plating dishes, portioning food and serving drinks or trays, captured by paid, consented contributors to a written skill spec, with synchronized streams, 3D hand pose, contact events and action segments.
Which plating and serving tasks does Firsthand list?
Eleven catalog rows: plating (Banquet plating, Plate + garnish, Cooking / Line Cooking), portioning (four rows) and serving (Pour + serve beverage, Tray assembly + delivery, Counter service and packaging, Order assembly). Statuses are Ready to deploy, Active collection or Available on request.
What streams and labels come with a plating episode?
Ten hardware-synchronized streams aligned under 2 ms, 3D hand pose for both hands, contact events and action segments with a start, an end, a verb and a noun. No dish type, portion weight, presentation label, force or tactile reading is stated.
How is privacy handled in kitchens and dining rooms?
Every contributor signs a written release, and faces and screens are blurred irreversibly, with a reviewer confirming no un-consented person is identifiable. Footage with bystanders who cannot be de-identified is discarded. How guests are handled in a dining room is settled in the spec.
How fast can a custom plating and serving dataset be collected?
From a signed spec to a first validated batch, a median of 48 hours where coverage is deep and three to four weeks where it is thin. The site does not say which applies to these rows, so ask when you request the quote.
Do you capture robot or teleoperation data for plating?
No. Firsthand captures humans, not robots. Episodes carry human hand pose and human action labels, so using them for a specific robot means your own retargeting step.
Who owns the data?
The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity as a separate term. The executed order form governs.

Sources

Where are the delivery formats on this page documented?

Request a custom plating and serving dataset.

Tell us the setting, the steps and the privacy rule your model needs to see. We will quote reach, timeline and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.