Task family

Egocentric Hospital and Clinical Care Task Data

Equipment setup, specimen handling, patient room prep, surface decontamination and procedures on a phantom or mannequin, captured first-person against a spec you write, for hospital robot training data, clinical care task video datasets and fast, on-demand custom healthcare task collection.
Streams
Ten, synced under 2 ms
Catalog status
Available on request
Billing
Per validated hour
License
Buyer-owned

Requirements

What does hospital task data need?

01

Named hospital tasks, not one generic label

Equipment setup, specimen handling, patient room prep, surface decontamination and procedures on a phantom are five different jobs. A spec names each one rather than relying on a single clinical label.

02

Both hands and the objects in frame

These tasks put two hands on monitors, pumps, specimen containers, linens and wipes. A head-mounted view keeps the hands and the objects in view through each step.

03

Hand pose and step boundaries on one clock

3D hand pose with a visibility flag per joint and frame-accurate action segments are aligned to video and depth on one hardware clock.

04

A scene boundary settled before capture

The site states a simulated-patient-only boundary for healthcare settings and never captures real patient footage. Who plays the patient is agreed in the written spec.

Why it matters

What is egocentric hospital and clinical care task data?

Egocentric hospital and clinical care task data is first-person video of hospital tasks: positioning and connecting bedside equipment, labelling and bagging specimen containers, preparing a patient room, wiping down surfaces, and practising clinical procedures on a phantom or mannequin. It is recorded from the operator viewpoint, with depth, 3D hand pose, contact events and frame-accurate action segments aligned on one hardware clock.

The site lists these as five rows in one environment, Hospital & Clinical Care, and states no other hospital row. This page covers those five and describes what is collected for robot training, not how to deliver care.

Setting

Which hospital and clinical care tasks does the site state?

The task catalog states five rows in the Hospital & Clinical Care environment, which sits in the Healthcare & Life Sciences sector. The rows are Equipment setup, Medical Procedures, Patient room prep, Specimen handling and Surface decontamination. Two are rated Easy, two Medium and one, Medical Procedures, Hard. All five are listed as Available on request.

The environment carries the examples "Patient room prep, equipment setup, specimen handling, surface decontamination -- requires HIPAA compliance". Those are the catalog examples for the environment. The site does not name a hospital, a ward or a provider.

The first table below lists each row exactly as the catalog words it, with its rating, capture mode and status.

The rows

What does each row say, step by step?

Equipment setup reads "Position and connect monitors, pumps and stands at the bedside and check they run." Patient room prep reads "Make the bed, stock supplies, position equipment and set up the room for the next patient." The site states the physical steps only. It names no device model and no clinical setting for the equipment.

Specimen handling reads "Label, bag and log specimen containers and place them in transport racks." Surface decontamination reads "Wipe down beds, rails and equipment with disinfectant and dispose of used wipes." The site names no disinfectant, no label format and no log system, and it makes no claim about correct infection-control practice.

Medical Procedures reads "Perform clinical procedures on a phantom or mannequin, dressings, injections and checks." The second table below sets each row against what the site does and does not state.

Scope

Does the site state clinical outcomes, medical decisions or patient care?

No. The five rows describe physical tasks, and the site states no diagnosis, treatment, assessment or clinical decision for any of them. This page makes no clinical-outcome, medical or patient-care claim beyond the row wording quoted above, and nothing here is clinical guidance.

One row sits closer to the edge than the others. The healthcare data page describes its task set as deliberately non-clinical, and says the site does not perform or claim to perform later-stage clinical validation. Medical Procedures is a catalog row for clinical procedures on a phantom or mannequin, rated Hard. The row wording is the only description the site gives, so exactly which steps a collection records is settled in the written spec.

Who is in the scene

Are real patients recorded, and what does the HIPAA wording mean?

No. The healthcare data page states that capture in healthcare settings happens only in training environments with simulated patients, and that real patient footage is never captured, under any circumstance, regardless of what a spec requests. The Medical Procedures row itself says phantom or mannequin. The site does not say which of those, or which person, stands in for the patient on the other four rows, so that is settled in the spec.

The catalog example text for this environment ends with the words "requires HIPAA compliance". That is catalog wording and nothing more. The site states no HIPAA certification, no compliance programme and no audit for these rows, and this page claims none. Because capture uses simulated patients only, the spec is where any regulatory question for your own program is raised.

Environments

Where are hospital tasks captured, and is it real work or a staged set?

The five rows sit in the Hospital & Clinical Care environment. Each row is Available on request, which the site defines as scoped and recruited to spec when a buyer asks for it. Every row names the capture mode Phone (head-mounted) / multiview rig, which the site describes in general as the first-person view of real work on site rather than the staged handheld-gripper mode.

For healthcare that has to be read with the simulated-patient boundary above, so this page does not claim work with real patients. The site also states that clinical and care settings are among its thinner-coverage environments, so access, sites and schedule are agreed in the written spec. The site names no facility.

Capture and labels

How is hospital work captured and what does each episode contain?

A head-mounted phone records the first-person view, and a multiview rig adds synchronized third-person angles when the spec asks for them. A wrist-mounted camera is available alongside the head camera for close-range detail. Each episode follows the episode schema used across Firsthand custom collections: ten streams synced under 2 ms, with head and wrist RGB, depth, 3D hand pose with per-joint visibility, contact and grasp events, and frame-accurate action segments.

For bedside tasks, contact events mark when the fingers make and break contact with a stand, a container or a wipe, and action segments mark where one step ends and the next begins. The site does not state a camera count, resolution, frame rate or any measurement of force or pace, so this page makes no precision claim.

Adjacent environments

How do the hospital rows differ from other healthcare environments?

The catalog has neighbouring healthcare environments: Nursing Home & Assisted Living (five care-support rows), Care Room (assistive care) (a staged set with twelve rows), Wet Lab & Clinical Bench (laboratory work) and Dental Lab & Medical Device Workshop. The site does not describe any of them as hospital work, so this page counts only the five Hospital & Clinical Care rows. Which environment fits is set in the spec.

Robots

Why does hospital task data matter for robot learning?

Hospital support tasks are sequences of manipulations in a set order, performed with two hands around monitors, pumps, containers, linens and wipes. First-person data gives the hand and object detail for each move, and the action segments give the order.

The site states no robot results for these rows, makes no claim that any model was trained on this data, and makes no safety claim about robots working near patients. This page describes what is collected, not what a policy trained on it will do.

Privacy

How is privacy protected?

Every episode links to a signed contributor release through a documented consent chain, and the consent record is part of the delivery. Contributors are the people who perform the tasks, so their release covers the footage captured of their hands and workspace.

Faces, plates and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. Clips that cannot be de-identified are discarded. The anonymization record is delivered with the data.

The site does not state how the screens category applies to a bedside monitor display, and it does not list wristbands, specimen labels or chart text as blur categories, so this page does not claim they are blurred. Treat them as spec items and settle in the spec how the scene is set up.

Delivery

How is hospital task data labeled and delivered?

Delivery formats are RLDS, LeRobot and WebDataset, plus HDF5, zarr and Rerun .rrd. The first three are the training-loader formats; HDF5 and zarr carry arrays for custom pipelines, and Rerun .rrd gives a multi-stream synchronized playback file for review. The last table below sets each against what its official documentation states.

Every batch is validated against signed accept criteria before it ships, with a reject log, and delivery is rolling, per validated batch.

Fast and on demand

How fast can a custom hospital task dataset be collected?

Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front.

The site lists clinical settings among the thin environments. The five rows are Available on request, and the site does not map that status to a specific number of days or state a hospital-specific contributor count, so the timeline is quoted against your spec. On-demand top-ups run against the same spec and accept criteria after each training run.

The spec

How do you start a pilot?

You describe the hospital tasks your policy needs: which of the five rows, the objects, the start state and the end state, and who or what plays the patient. We turn that into a written spec covering tasks, environments, capture tier, annotation depth, the scene boundary and accept criteria, then quote reach, timeline and price against that document before any capture begins.

A Spec Pilot, a small and fast first batch, can prove the spec and the accept criteria before you scale volume. Billing is per validated hour, a reject log ships with every batch, and rejected items are recaptured at our cost.

At a glance

What you get.

Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.

The stated rows

How does the catalog state the five Hospital & Clinical Care rows?

Every row in the Hospital & Clinical Care environment, as listed in the task catalog.
Catalog taskCatalog descriptionSkillCaptureStatus
Equipment setupPosition and connect monitors, pumps and stands at the bedside and check they run.MediumPhone (head-mounted) / multiview rigAvailable on request
Medical ProceduresPerform clinical procedures on a phantom or mannequin, dressings, injections and checks.HardPhone (head-mounted) / multiview rigAvailable on request
Patient room prepMake the bed, stock supplies, position equipment and set up the room for the next patient.EasyPhone (head-mounted) / multiview rigAvailable on request
Specimen handlingLabel, bag and log specimen containers and place them in transport racks.MediumPhone (head-mounted) / multiview rigAvailable on request
Surface decontaminationWipe down beds, rails and equipment with disinfectant and dispose of used wipes.EasyPhone (head-mounted) / multiview rigAvailable on request

Step by step

What is stated about each row, and what is not?

The steps in each row wording, with the details the site does not state and a spec would define.
RowWhat the site statesWhat the site does not state
Equipment setupPositioning and connecting monitors, pumps and stands at the bedside, checking they runDevice models, what running means, any clinical setting
Medical ProceduresClinical procedures on a phantom or mannequin: dressings, injections, checksWhich steps are recorded, phantom types, any clinical outcome
Patient room prepMaking the bed, stocking supplies, positioning equipment, setting up for the next patientBed types, which supplies, what counts as a prepared room
Specimen handlingLabelling, bagging and logging containers, placing them in transport racksContainer types, label formats, log system, any laboratory rule
Surface decontaminationWiping beds, rails and equipment with disinfectant, disposing of used wipesDisinfectant, wipe types, any infection-control standard

Streams and labels

What does each episode carry, and what is not stated?

Streams and labels the site states for Firsthand custom collections, and hospital task fields the site does not state.
ItemStatus on this siteWhy it matters for hospital tasks
Ten streams synced under 2 msStatedA step boundary lines up with the exact video and depth frames
Depth, with distance-invalid pixels flaggedStatedUnreliable pixels are marked, not interpolated
3D hand pose with per-joint visibilityStatedRecords the fingers through each step
Contact and grasp eventsStatedMarks when the hands take and release a container, a stand or a wipe
Frame-accurate action segmentsStatedMarks where one step ends and the next begins
Who or what plays the patient, and consent for that personNot statedSettled in the spec before capture
Device models, force, pace or task durationNot statedDefined in the spec if needed
Clinical outcomes, HIPAA certification, safety or robot resultsNot statedNo claim is made

Formats

Which formats carry hospital task data?

Delivery formats stated on the site, set against what each format official documentation states where the documentation was read.
FormatWhat the official documentation statesUsed for here
RLDSA dataset is retrieved as episodes, and each episode holds a dataset of steps (RLDS repository README)Episode-based training loaders
LeRobotLeRobotDataset v3.0 is a standardized format for robot learning data, including multi-camera video, with many episodes stored per Parquet or MP4 file (LeRobot documentation)LeRobot dataset format and tooling
WebDatasetTar files in which files that belong together share a basename, split into numbered shards (WebDataset README)Sharded streaming for large training runs
HDF5 and zarrStated by Firsthand onlyArray storage for custom pipelines
Rerun .rrdStated by Firsthand onlyMulti-stream synchronized playback for review

FAQ

Hospital robot training data, answered.

What is egocentric hospital and clinical care task data?
It is first-person video of hospital tasks such as equipment setup, specimen handling, patient room prep, surface decontamination and procedures on a phantom or mannequin, captured against a written spec with 3D hand pose, depth and action segments. Firsthand handles contributors, capture, QA, consent, and delivery.
Which hospital tasks does the site state?
Five rows in the Hospital & Clinical Care environment: Equipment setup, Medical Procedures, Patient room prep, Specimen handling and Surface decontamination. Two are Easy, two Medium and Medical Procedures is Hard, and all five are Available on request. The site states no other hospital row.
Does this include real patients?
No. The healthcare data page states that capture uses training environments with simulated patients only and never captures real patient footage. The Medical Procedures row names a phantom or mannequin. Who plays the patient on other rows is settled in the written spec.
Does the site claim HIPAA compliance or clinical outcomes?
No. The catalog example text for the environment says "requires HIPAA compliance", and that is wording only. The site states no certification, no clinical outcome, no safety result and no robot result for these rows, and nothing here is clinical guidance.
How fast can a custom hospital task dataset be collected?
Where coverage is deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, and the site lists clinical settings among them. The five rows are Available on request, so the timeline is quoted against your spec.
How do you start a pilot?
Describe the hospital tasks you need: which rows, the objects, where an episode starts and ends, and who or what plays the patient. We write a spec, quote reach, timeline and price against it, and a Spec Pilot proves the accept criteria before you scale. Billing is per validated hour under a perpetual, buyer-owned license.

Sources

Where are the delivery formats documented?

Request a custom dataset.

Tell us which hospital tasks your policy needs to handle and who or what plays the patient. We will quote reach, timeline, and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.