Now recruiting capture teams worldwide. Get paid to record first-person video of real work.

Apply to capture

Task rows

Stereo Table Setting and Clearing Data

Laying a full place setting, then stacking dishes, bussing a table and wiping it down, is short, repeated two-handed work where plates, glasses and cutlery hide the hands and sit close to each other on one surface. This page covers the stereo and container side of table setting robot training data for the two catalog rows that name this work: what a calibrated stereo pair adds to stereo place setting video, what MCAP is according to its own documentation, and what the site does not state. We run custom hospitality data collection fast and on demand against a spec you write, as a handheld gripper table bussing dataset when the spec names it, delivered as MCAP, as custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Catalog rows
"Set table service" (Medium) and "Clear + bus table" (Easy); Available on request
Environment
Hospitality Service Set (table, bar, room)
Container
MCAP (.mcap), open source
License
Perpetual, buyer-owned

Why stereo and MCAP here

What do a stereo pair and an MCAP file add to table setting and clearing video?

01

Depth across a crowded table surface

The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the hands, the ware and the table under them sit in one frame. The site gives no depth accuracy figure for these rows and does not say how depth behaves on glassware or polished plates.

02

Fingers behind a stack, flagged not guessed

The site states that fingers disappear behind objects. 3D hand pose carries a per-joint visibility flag, so a finger behind a stack of dishes is marked as hidden rather than silently filled in.

03

Two named rows, worded as the catalog words them

"Set table service" reads "Lay a full place setting." and "Clear + bus table" reads "Stack dishes, bus, and wipe." A spec names those rows and says where a run starts and ends, rather than relying on a generic dining label.

04

Several streams in one container

MCAP stores multiple channels of timestamped, pre-serialized data, so a stereo pair, hand pose and IMU can share one file on one log-time axis. Which streams go in the file is set in the skill spec.

Definition

What is stereo table setting and clearing data?

It is first-person video of a person laying a place setting and of a person stacking dishes, bussing and wiping a table, recorded with two calibrated cameras, with hand pose and sensor streams alongside. People reach it through several phrasings: table setting robot training data, stereo place setting video, a handheld gripper table bussing dataset, or custom hospitality data collection at scale. They describe one request from different angles. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on these rows. The catalog lists a multiview rig with handheld grippers as the capture setup, so a stereo pair is a capture choice agreed in the skill spec through the site’s general stereo offer and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data, so using it for a specific robot still needs your own retargeting step. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Catalog rows

Which catalog rows does this page rely on, and what is their status?

Two. "Set table service" and "Clear + bus table" both sit in the Hospitality Service Set (table, bar, room) environment of the Hospitality & Events sector, which the catalog describes with the examples a table, a counter or bar, and a housekeeping or room area. The first reads "Lay a full place setting." and is rated Medium. The second reads "Stack dishes, bus, and wipe." and is rated Easy. Each lists the capture setup "Multiview rig + handheld grippers" and has the status Available on request.

Available on request means the work is scoped and recruited to spec when a buyer asks. It is not a statement that footage of these tasks has been delivered, and the site states no delivered hours or stereo clips for them, so this page claims none. The first table below reproduces both rows exactly as the catalog states them. The site names no place setting layout, ware type, item count or table size for either row, so none is stated here.

The setting

What is the Hospitality Service Set environment?

The catalog describes it as a staged working set for hospitality service tasks. Beside the two rows on this page sit Fold napkins ("Fold napkins to a set style.", Medium) and Tray assembly + delivery ("Arrange, carry, and set a tray.", Easy), among others. The neighbouring rows are context only, not steps of the two table rows.

The site describes this capture mode as a staged working set recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector. The site states that Firsthand does not capture at a customer’s own facility, so the set is a capture setting arranged for the task, not a restaurant or a hotel dining room. Whether a stereo pair would be worn, mounted or combined with the multiview rig the catalog names is not stated, and is settled in the spec.

Difference

How is this different from the egocentric table setting, catering and event venue pages?

This page is the stereo and MCAP treatment of two hospitality service set rows, Set table service and Clear + bus table. The egocentric table setting and clearing page is the first-person page for the same two rows: stereo and MCAP are not its subject. It is a sibling, not an equivalent, and the two share the row wording.

The stereo catering and banquet plating page covers three Catering & Event Kitchen rows, all Available on request, with a head-mounted phone or multiview rig as the listed capture: banquet plating, bulk food preparation and event service setup and breakdown. The stereo event venue setup and AV cable taping page covers eight Event Venue rows, all Ready to deploy, for staging, cables, tables and chairs and sweeping. Neither is about laying one place setting or bussing one table. The comparison table below sets the pages side by side.

Capture setup

What capture setup do the rows list, and where does stereo come in?

The catalog lists "Multiview rig + handheld grippers" for both rows. The site does not state a stereo pair for them, and it does not describe these tasks as stereo capture. Stereo applies through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The site does not state a camera count for the rows.

The site describes two stereo setups and does not say which one an order for these rows uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output. Whether either is used with the catalog’s multiview rig is not stated.

Handheld grippers

Why does a handheld gripper capture matter for table bussing?

The catalog lists handheld grippers because the operator holds a gripper rather than a bare hand while performing the task, and the site says this lets demonstrations map onto a robot end effector. For a place setting or a bussed table, the grasp, the lift and the placement are the actions a policy has to reproduce. A handheld gripper table bussing dataset keeps the demonstration closer to the hardware a robot would use than a bare-hand clip does.

The site does not state which gripper is used or how it attaches, and this page names none. It also does not claim that a model trained on this capture succeeds at the task. The embodiment mapping is a design choice agreed in the skill spec, and the site does not claim that human capture replaces robot data.

Why stereo

Why capture place settings and table bussing in stereo?

Table service is about where each item lands relative to the others and to the table edge, and about how a stack is held while the other hand clears. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the geometry sits in the same capture that shows the hands working. The site gives no depth accuracy figure for these rows, so what a buyer does with that depth is their pipeline’s question.

A stereo pair adds depth, not a verdict. The site lists no label for a correctly laid place setting, a per-item position on the table, a breakage or a clean wipe for these rows, and none is claimed. The egocentric table setting and clearing page already supplies the first-person framing; stereo is the addition for a buyer who wants the depth and the MCAP file.

Steps

Which steps of setting and clearing a table can a stereo spec label?

The rows name four actions: laying a full place setting, stacking dishes, bussing, and wiping. A spec that says "clear the table" gives one long action segment. A spec that names the jobs gives a sequence you can slice by job. The site’s skill-spec guide says to name the verb and the noun rather than a broad activity, so the rows’ own verbs are the place to start.

Whether the jobs are separate segments, or a whole run is one segment, is a spec decision, not a default. The site does not set an episode length, so the spec says whether one episode is one place setting, one table or a laying run followed by a clearing run. The spec decides what counts as the ware, what counts as a place setting and where each job starts and ends, because the site names no layout or ware type for either row.

Occlusion and depth

What do stacked dishes and a crowded table do to visibility and depth?

Both hands are often busy in these rows, and a stack held in one hand can hide the fingers of the other. The site states that fingers disappear behind objects. That is why each hand joint carries a visibility flag, why contact events mark when the hand takes or releases an item, and why instance masks are recorded per frame.

The site also states that depth degrades on wet and reflective surfaces and that glare-heavy, low-texture scenes make single-frame depth worse, with per-frame confidence maps masking low-confidence regions. A wet table after wiping may be one place this matters, but the site does not say how depth behaves on glassware, polished plates or cutlery for these rows. If your policy depends on those frames, say so in the spec and settle it on a Spec Pilot.

Privacy

What does the site say about privacy and bystanders in a hospitality set?

For every delivered episode the site states that faces and screens are blurred irreversibly, that a reviewer confirms no un-consented person is identifiable, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data, an undocumented bystander is a stated reject criterion, and every episode links to a signed contributor release. Whether printed text on an item in view is blurred is not stated, so it is agreed in the spec.

The site also states that front-of-house hospitality settings are bystander-dense, which lowers yield after de-identification, while it classes room work as back-of-house. Which setting a table job is recorded in is agreed in the spec. This page gives no guest, venue or compliance claim beyond that. This is not legal advice, and the executed agreement controls.

MCAP

What is MCAP, per its own documentation?

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Streams

Which streams can ship in an MCAP delivery of table service data?

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For these rows the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Failure modes

What goes wrong when you record table service in stereo, and what does QA reject?

Hand occlusion by a stack or by other ware is the first risk, under the visibility flag above. Weak depth on wet, glare-heavy or reflective surfaces is the second. Calibration is measured at the start and end of every session under a 0.28 px RMS gate, and an episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, for the manipulated object out of frame for more than 15 percent of the episode, for an undocumented bystander or for a mismatch against the named skill spec.

A reject log ships with every batch, and rejected items are recaptured at the provider’s cost. For these rows, the spec decides what the manipulated object is: the plate, the glass, the stack in hand or the cloth wiping the table.

Scale and speed

How fast can custom stereo table setting data be collected at scale?

The rows are Available on request, so this is custom collection, not an off-the-shelf purchase. The off-the-shelf stereo and hand-tracking video can still show the format before you write a spec.

It is custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is 48 hours. The site classes hospitality as a thinner-coverage environment, where a batch typically takes three to four weeks, and the timeline is quoted against your spec. A pilot reaches first delivery in two to three weeks and a standing program reaches full rate in four to six weeks, as program milestones that assume coverage is in place. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Delivery and license

Which formats and license come with a stereo table setting order?

Delivery is MCAP, alone or beside the formats your stack already reads: RLDS, LeRobot, WebDataset, HDF5 or zarr, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader.

The license is perpetual, irrevocable and worldwide, with the data buyer-owned and non-exclusive by default. Training, evaluating, fine-tuning and deploying models, including commercial ones, is permitted, and exclusivity is available as a separate term. The executed agreement controls. The site does not publish a price list, a minimum order or a batch size for these tasks, so those are set in your quote.

Ordering

How do you order a stereo table setting and clearing dataset?

Describe the jobs and where each starts and ends: lay a full place setting, stack dishes, bus and wipe. Say whether an episode covers one place setting, one table or a laying run followed by a clearing run, what counts as the ware and what counts as a finished job, the privacy rule for the space, and name the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside a training format.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The last table below shows what each spec element decides.

At a glance

What you get.

Catalog row: Set table service
"Set table service" (Hospitality Service Set); Medium; Available on request
Catalog description
"Lay a full place setting."
Catalog row: Clear + bus table
"Clear + bus table" (Hospitality Service Set); Easy; Available on request
Catalog description
"Stack dishes, bus, and wipe."
Catalog capture setup
Multiview rig + handheld grippers on both rows. The site states no stereo pair for these rows, so a stereo pair is agreed in the skill spec through the general stereo offer
Stereo head rig (06)
Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAP
Hand pose
21 joints per hand with a per-joint visibility flag
Sync
Hardware-triggered onto one clock; 2.0 ms cross-stream ceiling
Language data
One task instruction per episode
Place setting layouts, ware types, item counts, table sizes, cycle times
Not stated beyond the row wording; agreed in the skill spec
MCAP topics, schemas, rates
Not stated on the site; agreed in the skill spec
License
Perpetual, irrevocable, buyer-owned, non-exclusive by default

Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.

Catalog rows

Which catalog rows cover table setting and clearing, and what is their status?

The two rows exactly as the task catalog states them, followed by neighbouring rows shown for context only. Items the catalog does not list are marked Not stated. Source file: lib/commercial-catalog-data.ts.
Task (as listed)Description (as listed)EnvironmentSkillCapture setupStatus
Set table serviceLay a full place setting.Hospitality Service Set (table, bar, room)MediumMultiview rig + handheld grippersAvailable on request
Clear + bus tableStack dishes, bus, and wipe.Hospitality Service Set (table, bar, room)EasyMultiview rig + handheld grippersAvailable on request
Fold napkins (context only)Fold napkins to a set style.Hospitality Service Set (table, bar, room)MediumMultiview rig + handheld grippersAvailable on request
Tray assembly + delivery (context only)Arrange, carry, and set a tray.Hospitality Service Set (table, bar, room)EasyMultiview rig + handheld grippersAvailable on request
Place setting layouts, ware types, item counts, outcome labelsNone beyond the row wordingNot stated as rowsNot statedNot statedNot stated

Compared pages

How does this page differ from the egocentric, napkin, tray, catering and event venue pages?

Each page and what it covers. Statuses are as the site lists them; none of these pages is claimed to hold delivered hours here.
PageCaptureWhat it coversRelation to this page
This page: stereo table setting and clearing dataStereo, agreed in the specSet table service (Medium); Clear + bus table (Easy); both Available on requestStereo pair and MCAP delivery for two hospitality service set rows
Egocentric table setting and clearing dataEgocentric, not stereoThe same two rows, first-personThe first-person sibling; stereo and MCAP are not its subject
Stereo napkin and towel folding dataStereo, agreed in the specFold napkins; Fold towels + linens (Medium, Available on request)Neighbouring rows in the same environment
Egocentric tray assembly and delivery dataEgocentric, not stereoTray assembly + delivery (Easy, Available on request)Neighbouring row in the same environment
Stereo catering and banquet plating dataStereo, agreed in the specThree Catering & Event Kitchen rows, all Available on requestDifferent scope: plating, bulk prep and service setup in a catering kitchen
Stereo event venue setup and AV cable taping dataStereo, agreed in the specEight Event Venue rows, all Ready to deployDifferent task: staging, cables, tables and chairs, sweeping
Egocentric stereo data in MCAPStereo, hubThe stereo and MCAP hub across tasksGeneral stereo and MCAP treatment; not specific to table service

Stated vs not stated

What the site states about stereo capture for table setting and clearing

Left to right: the item, what the site states, and what that means for a table service order. “Not stated” means the site does not say.
ItemWhat the site statesFor a table service order
Catalog capture setupMultiview rig + handheld grippers on both rowsNo stereo pair is stated for these rows; a stereo pair is a spec choice through the general stereo offer
Stereo head rig (dataset 06)Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAPWhether an order uses it: Not stated
Hard-environments stack4K/60 head camera, 120° lens; two 1080p/60 wrist cameras; depth 848×480 at 30 fps; 200 Hz 6-DoF IMUWhether an order uses it: Not stated
Hands, contacts and masksPer-joint visibility flag on hand pose; contact events; per-frame instance masks stored at 848×480Stated; what counts as the object is set in the spec
OcclusionFingers disappear behind objectsStated; how often it occurs on these rows is Not stated
Depth on wet and reflective surfacesDepth degrades on wet and reflective surfaces; confidence maps mask low-confidence regionsStated; behaviour on glassware and plates is Not stated
CalibrationMeasured every session and shipped with the episode; 0.28 px RMS reprojection gateStated
SynchronizationHardware-triggered onto one clock; 2.0 ms cross-stream ceilingStated
Row statusMedium and Easy, Available on requestNo volume, hours or delivered stereo footage is claimed on this page
Place setting layouts, ware types, item counts, table sizes, timesNot stated beyond the row wordingNot claimed; set in the spec
Layout correctness, breakage or outcome labelNot stated; no such label is listed for the rowsNot claimed; set in the spec
Task duration, success rateNot statedNot claimed on this page
MCAP topics, schemas, ratesNot statedAgreed in the skill spec

Stream map

Streams for table service work and what the site states about an MCAP file

Left to right: the stream or item, what the site states about it, and what the site states about it in MCAP. “Not stated” means the site does not say.
Stream or itemWhat the site statesIn an MCAP delivery, as the site states it
Stereo pairDual fisheye, 1080p per eye, 62 mm baseline (dataset 06)Dataset 06 is stated as shipped as MCAP; channel layout not stated
Depth with confidence mapsRecovered from the calibrated stereo pair; confidence map masks low-confidence regionsNot stated
Hand pose21 joints per hand with a per-joint visibility flagChannel layout and joint order not stated
IMU9-axis (dataset 06); 6-DoF at 200 Hz (hard-environments stack)Channel layout and rate not stated
Contact events, action segments, instance masksListed in the site episode schema; masks stored at 848×480Not stated
Task instructionOne per episodeNot stated
CalibrationMeasured every session and shipped with the episodePlacement in the file not stated
Topic names, schemas, encoding, ratesNot statedAgreed in the skill spec

Skill spec

What the skill spec fixes for a stereo table setting and clearing order

Spec element and where a buyer can see it in the delivery. “Set per program” means the site states no fixed value.
Spec elementWhat it decidesWhere it shows up
RowsSet table service and Clear + bus table, worded as the catalog words themSet per program; Available on request as listed
Jobs and orderWhat one job is: lay a place setting, stack dishes, bus, wipeEpisode boundaries and action segments; set per program
Episode scopeOne place setting, one table, or a laying run followed by a clearing runSet per program; the site does not state an episode length
Object definitionWhich item in frame counts as the object: the plate, the glass, the stack in hand or the clothContact events and the out-of-frame reject rule; set per program
Rig and streamsStereo pair, multiview angles, wrist views, hand pose, IMU and any othersThe channels you can read; confirm on the Spec Pilot
Setting and lightingWhere capture runs and how the set is litSet per program; the site states no lighting setup for these rows
MCAP topics and encodingWhat your reader expectsSet per program
Delivery formatHow the data is packagedMCAP alone, or beside RLDS, LeRobot, WebDataset, HDF5 or zarr
Accept criteriaWhat passes QA, including what counts as a rejectReject log per batch
PrivacyHow people, screens and printed text in frame are handledFaces and screens blurred at ingest; other items agreed in the spec
LicenseWho owns the datasetPerpetual, irrevocable, buyer-owned, non-exclusive by default

Example footage

What does stereo capture look like on this site?

One real clip already published on this site, shown as a rig example. It is a head-mounted stereo capture with hands in frame. It does not show a place setting, a bussed table or a hospitality set, and it is not an example of the multiview rig with handheld grippers these rows list.

egocentric_stereo_sample.mp4 · head-mounted stereo · hands in frame · 1920 × 1080 web preview · rig example, not a table setting clip

No stereo table setting or table bussing clip is published on the site, so this is labelled as a rig example. Request a sample to see delivered files. See sample data outputs, stereo sample page, request an egocentric data sample.

FAQ

Table setting robot training data, answered.

What is stereo table setting and clearing data?
First-person video from two calibrated cameras of a person laying a full place setting, stacking dishes, bussing and wiping a table, with depth and hand pose alongside.
Which catalog rows cover it, and what is their status?
"Set table service" ("Lay a full place setting.", Medium) and "Clear + bus table" ("Stack dishes, bus, and wipe.", Easy) in Hospitality Service Set (table, bar, room). Both are listed with Multiview rig + handheld grippers and are Available on request. This page claims no delivered hours.
Is stereo already captured on these rows?
Not stated. The catalog lists a multiview rig with handheld grippers for both rows. A stereo pair is agreed in the skill spec through the general stereo offer and confirmed on a Spec Pilot.
How is this different from the egocentric table setting and clearing page?
The egocentric page covers the same two rows as first-person video. This page covers them with a stereo pair and MCAP delivery. The stereo catering and event venue pages cover different tasks.
Is there an off-the-shelf table setting dataset?
No. The rows are Available on request, so this is custom collection, at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Does the data include place setting layout or outcome labels?
Not stated. The site lists no place setting layout, ware type, item count, breakage or correctness label and no force reading, so none is claimed. These are set in the spec.
How fast can a custom stereo table setting dataset be collected?
From signed spec to first validated batch, a median of 48 hours where coverage is deep. Hospitality is a thinner-coverage environment that typically takes three to four weeks, so it is quoted against your spec.
Can it be delivered as MCAP, and what is in the file?
Yes. Firsthand offers MCAP delivery. Topic names, message schemas and rates are not stated on the site and are agreed in the skill spec.

Sources

Where is MCAP documented?

Request a custom stereo table setting and clearing dataset delivered as MCAP.

Tell us the jobs and where each starts and ends, whether an episode covers one place setting, one table or a laying run followed by a clearing run, what counts as the ware, the setting and lighting, the privacy rule for the space, the streams you want in the file and whether MCAP sits alone or beside a training format. We will quote reach, timeline and price against a written spec.