Three named tasks, not one care label
Carrying and placing a tray, positioning a drink with a straw and presenting a spoon are three different sequences, so a spec names each one, where it starts and where it ends.
Now recruiting capture teams worldwide. Get paid to record first-person video of real work.
Apply to captureCare room tasks
Requirements
Carrying and placing a tray, positioning a drink with a straw and presenting a spoon are three different sequences, so a spec names each one, where it starts and where it ends.
The catalog lists a multiview rig with handheld grippers for all three rows, so the spec states which streams that mode carries rather than assuming them.
Where the spec asks for them, contact events and action segments mark when a tray, a cup or a spoon is picked up, set down or handed on, aligned to video and depth.
The environment examples say participant or mannequin, and the site does not say which a given row uses. The spec settles it, under the simulated-patient-only boundary.
A perpetual, buyer-owned commercial license backed by a documented consent chain.
Definition
It is video of three tray and table tasks in a staged care room: carrying and placing a meal tray, positioning a drink with a straw, and presenting a spoon. People reach it through several phrasings: egocentric assisted living data, first-person meal tray and drink with straw video, an assisted feeding robot dataset, or custom care task data collection. All describe the same work, recorded against a written spec with depth, contact events and frame-accurate action segments aligned on one hardware clock where the spec includes them.
The elder care and assisted living page names two of these rows in a short paragraph about an adjacent environment, and the grooming page covers a different Care Room row. This page puts the three meal and drink rows in one place so a spec can be read at once. The data is human capture by paid, consented contributors, not robot or teleoperation data, so use for a specific robot still needs your own retargeting step.
Catalog rows
Three, all in the Care Room (assistive care) environment of the Healthcare & Life Sciences sector. "Deliver meal tray" is described as "Carry and place a meal tray." and rated Easy. "Serve drink + straw" is described as "Position a drink with a straw." and rated Easy. "Assisted feeding" is described as "Present a spoon to a participant/mannequin." and rated Medium.
All three are listed as Available on request, with the capture setup "Multiview rig + handheld grippers". Available on request means the work is scoped and recruited to spec when a buyer asks for it, rather than running as an open capture stream today. The site states no delivered hours for any of the three rows, so this page claims none.
The environment
The catalog describes the Care Room with the examples "Care room: bed, tray table, linens, participant or mannequin". It has twelve rows, all Available on request, and every one lists the capture setup Multiview rig + handheld grippers. Nine are outside this page: Arrange comfort items, Change bed linens, Fold + organize laundry, Grooming setup + handover, Hand over mobility aid, Make a bed, Sort medication packs, Tidy + reset room and Wipe + clean surfaces.
The site does not describe the Care Room as a facility and names no provider or site. It does not say whether a given row uses a participant or a mannequin, so this page says neither. It describes a staged room with a bed, a tray table and linens, and leaves who or what receives the tray, the drink or the spoon to the written spec.
Difference
The elder care and assisted living page is built on five rows in the Nursing Home & Assisted Living environment, and its capture mode is a head-mounted phone or multiview rig. It lists the Care Room rows only as an adjacent environment. The tray assembly and delivery page covers one row in a hospitality service set, and the meal plating and serving page covers plating, portioning and serving on the kitchen and service side.
The patient mobility page covers five mobility rows, including Hand over mobility aid, and the grooming page covers one Care Room row. A row called Feeding sits in Childcare & Early Education and a row called Feeding assistance sits in Nursing Home & Assisted Living; neither is part of this page. None of the pages has the three Care Room meal and drink rows as its main topic, which is why this page exists. The comparison table below sets them side by side.
Meal tray
One row, "Deliver meal tray", and one sentence: "Carry and place a meal tray." It holds two moves, a carry and a place, and it is rated Easy, which is all the site says about its difficulty. The environment examples list a tray table, but the row does not say that the tray is placed on it, so where it is placed is set in the spec.
What is on the tray, how heavy it is, how far it is carried, the route and what counts as placed are Not stated. A spec can decide whether an episode covers the carry, the placement or both, and the accept criteria say what counts as a finished step.
Drink and straw
One row, "Serve drink + straw", described as "Position a drink with a straw." and rated Easy. It names one move, positioning. The row is the one the site also lists among its drink rows on the beverage and coffee prep pages; this page treats it in its Care Room setting.
The drink, the container, the straw, the fill level, where the drink ends up and who or what it is positioned for are Not stated, and this page names none of them. A spec has to say what the object is, because the contact events and the out-of-frame rule both depend on it.
Spoon
One row, "Assisted feeding", described as "Present a spoon to a participant/mannequin." and rated Medium. It is the only one of the three that names a recipient, and the only one rated Medium.
The site does not state what is on the spoon, how close it is brought, how long a presentation takes or what happens after it. It makes no clinical, care or outcome claim for the row, and nothing on this page is guidance on how to assist anyone to eat. It describes what is collected for robot training.
Why demanding
The catalog describes steps that end in an object reaching a place or a person: a tray carried and placed, a drink positioned with a straw, a spoon presented. The rows are rated Easy, Easy and Medium, and this page says no more about difficulty than that. Lighting, room layout and reach are Not stated.
What the site does state is what egocentric video puts into the training distribution: hands entering the frame from below, occlusion during grasp and motion blur during reach. RGB alone leaves a model to infer 3D hand structure it cannot see reliably through occlusion, which is why the site delivers hand pose with a visibility flag per joint where a spec includes it. Whether that stream is populated for the staged gripper capture mode is Not stated.
Capture setup
The catalog describes the mode as "a staged working set recorded by a multiview rig while the operator performs the task with handheld grippers, so demonstrations map onto a robot end effector." It does not list a head-mounted phone or a stereo pair for any of the three rows, and this page does not claim either is running on them. The site states no camera count, resolution or frame rate for these rows, so the capture tier is agreed in the spec.
Stereo applies only through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. Whether the first-person head-mounted mode listed for other rows is offered for these three is Not stated. The site states no gripper model or jaw type, and care settings are among its thinner-coverage environments, so access and schedule are agreed in the spec.
Streams and labels
Each episode follows the Firsthand episode schema: ten streams synced under 2 ms, with RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments. For these rows, contact events mark when a tray, a cup or a spoon is taken up or released, and action segments mark each step as the skill spec defines it.
The site does not state which of those streams the staged gripper mode populates for each row. Object labels use the terms your spec sets. Force or tactile data, and success or quality labels for a placed tray, a positioned drink or a presented spoon, are Not stated. The streams table below sets each against what the site states.
Gloves
The site offers a tactile glove tier as a pilot-ready option. The glove signal is time-synced to a head-mounted stereo camera by hardware timestamps and an IMU, and each glove is calibrated every session.
The site does not state that the glove tier has been used on any Care Room row, and all three rows list handheld grippers as their capture mode. Treat force and tactile data as Not stated for these rows unless a spec adds the glove tier and agrees the fit with you.
People and privacy
No. The healthcare data page states that capture in healthcare settings happens only in training environments with simulated patients, and that real patient footage is never captured, under any circumstance, regardless of what a spec requests. The Care Room rows sit in the same sector, and the site does not separately say how that boundary names a participant, so who or what stands in for the recipient, and how that person is consented, is settled in the spec.
Every episode links to a signed contributor release through a documented consent chain. Faces, plates and screens are detected and irreversibly blurred at ingest, a reviewer confirms the result before anything ships, and the anonymization record is delivered with the data. An undocumented bystander is a stated reject criterion. How name tags, wristbands or room labels are handled is Not stated, so name them in the spec.
Quality gates
Episodes are machine-checked at ingest against the stated reject criteria, listed in the failure-mode table below, and a human reviewer then checks the result. Two stated rules bite on this work: an episode is rejected when the manipulated object is out of frame for more than 15 percent of it, and when more than 35 percent of depth pixels in the manipulation region are invalid.
What counts as the object, a tray, a cup with a straw or a spoon, is set in the spec. How often an object leaves the view, how depth behaves on a glossy tray or a clear cup, and any blur or sharpness rule are Not stated. A rejected item goes to the reject log and is recaptured at our cost.
Delivery
Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. Firsthand also offers MCAP delivery; the MCAP hub page covers the container as its own documentation describes it, and which streams would be channels in a file, and what they are named, is Not stated and is agreed in the skill spec.
Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch. The formats table below sets each format against what its official documentation states.
Scale and speed
None of the site’s off-the-shelf dataset descriptions names a Care Room meal, drink or feeding row, so for this task it is custom collection, not an off-the-shelf purchase. It is custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
From signed spec to first validated batch, the timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and operators need to be recruited and vetted first. All three rows are Available on request, so which range applies is confirmed when the spec is quoted rather than assumed. On-demand top-ups run against the same spec and accept criteria after each training run. Collections are billed per validated hour, and the site publishes no price list, minimum order or batch size for these rows.
Ordering
Describe which of the three rows are in scope, where each task starts and ends, whether an episode covers one move or a run of them, and what counts as finished. Add the setting, who or what receives the item, the privacy rule for the room, and the variation your pipeline is missing. Then name the streams you want, the capture mode and the delivery format.
Most programs begin with a Spec Pilot, a small, fast first batch that proves the spec and accept criteria on real footage before you scale. The last table below shows what each spec element decides.
At a glance
Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.
Catalog rows
| Task (as listed) | Description (as listed) | Environment | Skill | Capture setup | Status |
|---|---|---|---|---|---|
| Deliver meal tray | Carry and place a meal tray. | Care Room (assistive care) | Easy | Multiview rig + handheld grippers | Available on request |
| Serve drink + straw | Position a drink with a straw. | Care Room (assistive care) | Easy | Multiview rig + handheld grippers | Available on request |
| Assisted feeding | Present a spoon to a participant/mannequin. | Care Room (assistive care) | Medium | Multiview rig + handheld grippers | Available on request |
| Any other meal, drink or feeding step in the Care Room | Not stated as catalog rows | Not stated | Not stated | Not stated | Not stated |
Compared pages
| Page | Main topic | Capture framing | Treatment of the three rows |
|---|---|---|---|
| This page: egocentric care room meal and drink service data | Deliver meal tray, Serve drink + straw, Assisted feeding | Multiview rig + handheld grippers, as the catalog lists it | The whole page: scope, limits, streams and ordering |
| Egocentric elder care and assisted living data | Five Nursing Home & Assisted Living rows | Phone (head-mounted) / multiview rig | Names the Care Room rows only as an adjacent environment |
| Egocentric tray assembly and delivery data | One Hospitality Service Set row | Multiview rig + handheld grippers | Not covered; a different environment |
| Egocentric meal plating and serving data | Plating, portioning and serving rows | As the catalog lists them | Not covered; kitchen and service side |
| Egocentric patient mobility assistance data | Five mobility rows across three environments | As each row lists it | Not covered; mobility rows only |
| Egocentric grooming setup and handover data | One Care Room row, Grooming setup + handover | Multiview rig + handheld grippers | Not covered; a different row in the same environment |
Stated vs not stated
| Item | What the site states | For a meal and drink order |
|---|---|---|
| Catalog capture setup | A staged working set recorded by a multiview rig while the operator performs the task with handheld grippers | Stated for all three rows; no head-mounted phone or stereo pair is listed for them |
| Stereo | A stereo pair is a capture choice agreed in the skill spec through the general stereo offer | Not running on these rows as far as the site states; agreed in the spec |
| Participant or mannequin | Both appear in the environment examples; which a row uses is Not stated | Settled in the spec under the simulated-patient-only boundary |
| Camera count, resolution, frame rate for these rows | Not stated | Capture tier agreed in the spec |
| Gripper model and jaw type | Not stated | Agreed in the spec |
| Hand pose in gripper mode | Not stated | Agreed in the spec and confirmed on a Spec Pilot |
| Tray contents, drink, container, straw, spoon | Not stated | Not claimed; set in the spec |
| Faces and screens | Detected and irreversibly blurred at ingest; reviewer confirms; footage that cannot be de-identified is discarded | Stated; record ships with the data |
| Row status | Available on request | No volume, hours or delivered footage is claimed on this page |
Streams and tasks
| Stream or label | Status on this site | What it carries in a meal or drink episode |
|---|---|---|
| RGB and depth | Stated; ten streams synced under 2 ms | Keeps the tray, cup or spoon and the hands on one timeline |
| 3D hand pose, per-joint visibility | Stated; two hands of 21 joints, with a visibility flag per joint | Carries how the fingers hold an object and which joints were hidden |
| Contact and grasp events | Stated | Marks when a tray, cup or spoon is taken up or released |
| Action segments | Stated; frame-accurate | Separates the carry, place, position and present steps |
| Per-frame instance masks | Stated; stored at 848×480 | Follows the object set in the spec |
| Object labels | Set by your spec | Names the objects and steps in the terms your policy uses |
| Force or tactile data | Glove tier stated; use on these rows Not stated | Added only if a spec includes the glove tier |
Failure modes and QA
| Failure mode | What the site states | What a buyer sees |
|---|---|---|
| Fingers hidden behind an object | Per-joint visibility flag on hand pose where a spec includes it | A hidden joint flagged as hidden, not guessed |
| Object out of frame | Episode rejected when the manipulated object is out of frame for more than 15 percent of it | Reject log per batch; what the object is gets set in the spec |
| Invalid depth in the manipulation region | Episode rejected when more than 35 percent of depth pixels are invalid; behaviour on glossy or clear objects Not stated | Reject log per batch |
| Sync breach | Rejected beyond the ceiling for more than three consecutive frames | Reject log per batch |
| Calibration drift | Rejected beyond tolerance | Calibration shipped with the episode |
| Bystander in frame | Undocumented bystander rejected; faces and screens blurred and reviewer-confirmed | Reject log per batch; anonymization record ships with the data |
| Spec mismatch | Rejected for a mismatch against the named skill spec | Reject log per batch |
| Blur or sharpness | Not stated | Set in the spec if your policy needs a rule |
Formats
| Format | What the official documentation states | Used for here |
|---|---|---|
| RLDS | A dataset is retrieved as episodes, and each episode holds a dataset of steps (RLDS repository README) | Episode-based training loaders |
| LeRobot | LeRobotDataset v3.0 is a standardized format for robot learning data, including multi-camera video, with many episodes stored per Parquet or MP4 file (LeRobot documentation) | LeRobot dataset format and tooling |
| WebDataset | Tar files in which files that belong together share a basename, split into numbered shards (WebDataset README) | Sharded streaming for large training runs |
| MCAP | An open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data (MCAP site) | Multi-stream files; topics and schemas Not stated, agreed in the spec |
| HDF5 and zarr | Stated by Firsthand only | Array storage for custom pipelines |
| Rerun .rrd | Stated by Firsthand only | Multi-stream synchronized playback for review |
Delivery and licensing
| Item | As stated on this site | Note |
|---|---|---|
| Formats | RLDS, LeRobot, WebDataset, HDF5, zarr, Rerun .rrd, MCAP, or raw streams with metadata | Chosen in the skill spec; converter source ships with the data |
| License | Perpetual, irrevocable, worldwide, buyer-owned, non-exclusive by default | Exclusivity is available as a separate term; the executed agreement controls |
| Billing | Per validated hour | Rejected items are recaptured at our cost |
| Privacy | Faces, plates and screens blurred irreversibly; footage that cannot be de-identified is discarded | Anonymization record delivered with the data |
| Timeline | a median of 48 hours to first validated batch where coverage is deep; three to four weeks where thin | Which applies to these rows is Not stated; quoted against your spec |
| Price, minimum order, batch size | Not published for these rows | Set in your quote |
Skill spec
| Spec element | What it decides | Where it shows up |
|---|---|---|
| Rows in scope | Meal tray, drink and straw, spoon, or any mix | What the cameras see; set per program |
| Steps and order | What one step is and how steps are named | Episode boundaries and action segments; set per program |
| Episode scope | One move, one row or a longer run | Set per program; the site does not state an episode length |
| Object definition | Whether a tray, a cup with a straw or a spoon counts as the object | Contact events and the out-of-frame reject rule; set per program |
| Recipient | Participant or mannequin, and how it is consented | Settled before capture; simulated-patient-only boundary applies |
| Capture mode and streams | Multiview rig with handheld grippers, any stereo pair, wrist views and hand pose | The channels you can read; confirm on the Spec Pilot |
| Accept criteria | What counts as a finished step and what passes QA | Reject log per batch |
| Delivery format | How the data is packaged | RLDS, LeRobot, WebDataset, HDF5, zarr or MCAP |
| License | Who owns the dataset | Perpetual, irrevocable, buyer-owned, non-exclusive by default |
Example footage
One real clip already published on this site, shown as a format and rig example: a head-mounted stereo capture with hands in frame. It does not show a meal tray, a drink, a straw or a spoon, and it is not a multiview-rig or handheld-gripper clip.
No care room meal or drink clip is published on the site, so this is labelled as a format and rig example. Request a sample to see delivered files. See stereo sample page, request an egocentric data sample.
FAQ
Sources
Tell us which of meal tray delivery, drink and straw service and spoon presentation are in scope, where each task starts and ends, who or what receives the item, the capture mode, the privacy rule for the room, and the streams and format you want. We will quote reach, timeline and price against a written spec.