Depth for the hands and the objects they hold
The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the hands, the mannequin and the items in use sit in the same three-dimensional frame.
Now recruiting capture teams worldwide. Get paid to record first-person video of real work.
Apply to captureHard task
Why stereo and MCAP here
The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the hands, the mannequin and the items in use sit in the same three-dimensional frame.
An object or a mannequin can hide the fingers that hold it. 3D hand pose carries a per-joint visibility flag, so a hidden finger is marked hidden rather than filled in, and contact events mark when a hand meets an object.
Dressings, injections and checks are different activities. Streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling, and action segments mark where one step ends and the next begins.
MCAP stores multiple channels of timestamped, pre-serialized data, so a stereo pair, hand pose and IMU can share one file on one log-time axis. Which streams go in the file is set in the skill spec.
Definition
It is first-person video of a person carrying out clinical procedures on a phantom or mannequin, dressings, injections and checks, recorded from their own point of view with two calibrated cameras, with hand pose and sensor streams alongside. People reach it through several phrasings: stereo clinical procedure video, phantom and mannequin training robot data, a dressing and injection practice dataset, or custom clinical data collection at scale.
The site does not state that stereo is already running on this row. The catalog lists a head-mounted phone or a multiview rig as the capture setup, so a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot, through the site’s general stereo offer. The data is human capture by paid, consented contributors, not robot or teleoperation data. Firsthand is not affiliated with Foxglove or the MCAP maintainers.
Catalog row
One. "Medical Procedures" sits in the Hospital & Clinical Care environment, in the Healthcare & Life Sciences sector. Its description reads "Perform clinical procedures on a phantom or mannequin, dressings, injections and checks." It is rated Hard, it lists "Phone (head-mounted) / multiview rig" as the capture setup, and it is Available on request. The first table below reproduces it exactly.
Available on request means the work is scoped and recruited to spec when a buyer asks for it, rather than running as an open capture stream today. The site states no delivered hours and no delivered stereo footage for this row, so this page claims none. The row names a phantom or mannequin and three kinds of activity. It names no phantom type, no mannequin type, no dressing, no injection route and no check, so anything more specific is written into the spec and not assumed.
Difference
The egocentric clinical procedures on mannequins page covers the same row from the first-person side: the episode contents, the privacy handling and the delivery formats. The egocentric hospital and clinical care page is the family page, with five rows of which Medical Procedures is one. The stereo surgical instrument handling page covers a different row in a different environment, so it is a stereo neighbour and not a substitute.
This page narrows to the one Medical Procedures row and gives it a stereo and MCAP treatment: what a calibrated pair adds, which streams could ship in a file, and what is not stated. The site has no stereo hospital family page, so this is the only stereo page for this row. The second table below sets the pages side by side.
Hard
Hard is the skill level shown in the catalog, one of Easy, Medium or Hard. The site states no further criteria, so this page adds none. The only grounds for calling the row demanding are in its own wording: it bundles dressings, injections and checks into one row, it is performed on a phantom or mannequin, and the catalog rates it Hard while rating two of the other four hospital rows Easy and two Medium.
The site states no failure rate, tolerance, cycle time or skill level for the people who perform it, and this page makes no such claim. It also does not say that the row is harder to capture than another Hard row. What it does state is the mechanism that applies to any two-handed task: hand pose carries a per-joint visibility flag, and action segments mark where one step ends and the next begins.
Why stereo
A procedure on a mannequin is a sequence of two-handed moves on and around a body-shaped form. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the hands, the mannequin surface and the items in use share a frame and a scale.
The site gives no depth accuracy figure and no label for any clinical measurement, so whether a specific distance or contact is resolvable in depth is Not stated. How stereo depth behaves on glossy or translucent items, on a reflective mannequin surface or on a fast hand is also Not stated, and a spec should check it on a Spec Pilot. Capture at your own facility is listed as not offered.
Capture setup
The site describes two stereo setups and does not say which one this row uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output.
Which rig, resolution and streams an order for this row uses is Not stated, and the third table below sets what is stated beside what is not. Intrinsics and extrinsics are re-solved at the start and end of every session, and streams are hardware-triggered onto one clock with a 2.0 ms cross-stream ceiling. The catalog states that a multiview rig adds synchronized third-person angles when the spec asks for them, and wrist cameras give a closer view of the hands.
Who is in the scene
No. The healthcare data page states that capture in healthcare settings happens only in training environments with simulated patients, and that real patient footage is never captured, under any circumstance, regardless of what a spec requests. The row itself says phantom or mannequin. No patient data and no clinical outcome is part of the collection.
This page describes what is collected for robot training. It is not medical advice, not clinical training material and not guidance on how to perform a procedure. The site makes no clinical-accuracy, safety, sterility or outcome claim for this row. The catalog example text for the environment ends with the words "requires HIPAA compliance". That is catalog wording only: the site states no certification, compliance programme or audit for this row, and this page claims none.
MCAP
MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.
Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project’s licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.
Streams
Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For two-handed work the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments and per-frame instance masks elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.
No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.
Failure modes
Occlusion is the first risk. An item in the hand, a mannequin or the other hand can hide fingers, and the item can leave the view as it moves. The visibility flag on each joint is the answer the site gives, contact events mark when a hand meets an object, masks are recorded per frame, and a spec can ask for wrist views so a grip the head camera loses is still recorded.
Sequence is the second. Dressings, injections and checks are separate activities, so the spec decides how each is split into action segments. A spec that says only “do the procedure” gives one long segment. Lighting, glare, glossy surfaces and motion blur are Not stated for this row and are raised in the spec. Force and tactile readings are not among the listed streams of standard capture.
Calibration is the third, because stereo depth is only as good as the rectification behind it: calibration is measured at the start and end of every session under a 0.28 px RMS gate. An episode is also rejected when the manipulated object is out of frame for more than 15 percent of it, when more than 35 percent of depth pixels in the manipulation region are invalid, or when an undocumented bystander appears, and a reject log ships with every batch.
Not stated
Which steps are recorded, which phantoms or mannequins are used, any named dressing, injection route or check, the tools and materials beyond the words in the row, episode lengths, error rates, force or tactile readings, any success or completion label, any sterility standard, any licensing or regulatory status, and what a correct procedure means are Not stated. They are set in the written spec, and this page claims none of them.
The site also names no hospital, clinician, facility or client for this row. Who performs the task and where it is recorded are agreed in the spec. Faces, plates and screens are irreversibly blurred at ingest for every delivered episode, footage that cannot be de-identified is discarded, and any other item in frame is a spec item.
Scale
The site lists no off-the-shelf clinical procedure dataset, so this work is custom collection. The only stereo head rig the site states as shipping as MCAP is dataset 06, and this page does not claim off-the-shelf stereo coverage of clinical procedures. Custom collection covers the rest: custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks, and the site lists clinical settings among the thin environments. The row is Available on request, and the site does not map that status to a specific number of days, so the timeline is quoted against your spec. On-demand top-ups run against the same spec and accept criteria. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.
Ordering
Describe which of the three activities the row names you need, where each episode starts and ends, what phantom or mannequin stands in for the patient, the setting and lighting, and the privacy rule for anyone in frame. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.
Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The fifth table below shows what each spec element decides.
At a glance
Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.
Catalog row
| Task (as listed) | Description (as listed) | Environment | Skill | Capture setup | Status |
|---|---|---|---|---|---|
| Medical Procedures | Perform clinical procedures on a phantom or mannequin, dressings, injections and checks. | Hospital & Clinical Care | Hard | Phone (head-mounted) / multiview rig | Available on request |
| Stereo capture on this row | Not stated on the row; agreed in the skill spec | Not stated | Not stated | Not stated | Not stated |
| Named phantoms, mannequins, dressings, tools or procedures | Not stated | Not stated | Not stated | Not stated | Not stated |
Compared pages
| Page | What it covers | Treatment of Medical Procedures |
|---|---|---|
| This page: stereo clinical procedures on mannequins data | One row: Medical Procedures (Hospital & Clinical Care) | Stereo pair and MCAP delivery, stream map and failure modes for this row only |
| Egocentric clinical procedures on mannequins data | The same row, first-person | Episode contents, privacy and delivery formats; stereo is covered on this page |
| Egocentric hospital and clinical care task data | Five rows: Equipment setup, Medical Procedures, Patient room prep, Specimen handling, Surface decontamination | The family page; Medical Procedures is one of five |
| Stereo surgical instrument handling data | A different row in a different environment | Not covered; a stereo neighbour for a separate task |
| Egocentric stereo data in MCAP | The stereo and MCAP hub across tasks | General stereo and MCAP treatment; not specific to clinical procedures |
Stated vs not stated
| Item | What the site states | For a clinical procedures order |
|---|---|---|
| Catalog capture setup | Phone (head-mounted) / multiview rig on the row | A stereo pair is a spec choice; not stated on the row |
| Stereo head rig (dataset 06) | Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAP | Whether an order for this row uses it: Not stated |
| Hard-environments stack | 4K/60 head camera, 120° lens; two 1080p/60 wrist cameras; depth 848×480 at 30 fps; 200 Hz 6-DoF IMU | Whether an order for this row uses it: Not stated |
| Objects, hands and masks | Per-frame instance masks stored at 848×480; per-joint visibility flag on hand pose; contact events | Stated; which item counts as the object is Not stated |
| Calibration | Measured every session and shipped with the episode; 0.28 px RMS reprojection gate | Stated |
| Synchronization | Hardware-triggered onto one clock; 2.0 ms cross-stream ceiling | Stated |
| Scene | Training environments with simulated patients only | A phantom or mannequin; never a real patient |
| Row status | Available on request | No volume, hours or delivered stereo footage is claimed on this page |
| Force or tactile streams | Not among the listed streams of standard capture | Not claimed on this page |
| Clinical accuracy, safety, sterility, certification or robot results | Not stated | No claim is made |
| MCAP topics, schemas, rates | Not stated | Agreed in the skill spec |
Stream map
| Stream or item | What the site states | In an MCAP delivery, as the site states it |
|---|---|---|
| Stereo pair | Dual fisheye, 1080p per eye, 62 mm baseline (dataset 06) | Dataset 06 is stated as shipped as MCAP; channel layout not stated |
| Depth with confidence maps | Recovered from the calibrated stereo pair; confidence map masks low-confidence regions | Not stated |
| Hand pose | 21 joints per hand with a per-joint visibility flag | Channel layout and joint order not stated |
| IMU | 9-axis (dataset 06); 6-DoF at 200 Hz (hard-environments stack) | Channel layout and rate not stated |
| Contact events, action segments, instance masks | Listed in the site’s episode schema; masks stored at 848×480 | Not stated |
| Task instruction | One per episode | Not stated |
| Calibration | Measured every session and shipped with the episode | Placement in the file not stated |
| Topic names, schemas, encoding, rates | Not stated | Agreed in the skill spec |
Skill spec
| Spec element | What it decides | Where it shows up |
|---|---|---|
| Activities | Which of dressings, injections and checks are recorded, and in what order | Action segments; set per program |
| Stand-in for the patient | Which phantom or mannequin is used | Set per program; never a real patient |
| Episode boundary | One step, one procedure or a run of procedures | Set per program; the site does not state an episode length for this row |
| Rig and streams | Stereo pair, wrist views, multiview angles, hand pose, IMU and any others | The channels you can read; confirm on the Spec Pilot |
| MCAP topics and encoding | What your reader expects | Set per program |
| Calibration delivery | Inside the file or beside it | Set per program |
| Delivery format | How the data is packaged | MCAP alone, or beside RLDS, LeRobot, WebDataset, HDF5 or zarr |
| Accept criteria | What counts as a recorded step and what passes QA, including what counts as a reject | Reject log per batch |
| Privacy | How anyone, faces, screens and labels in frame are handled | Faces, plates and screens blurred at ingest; other items agreed in the spec |
| License | Who owns the dataset | Perpetual, irrevocable, buyer-owned, non-exclusive by default |
Example footage
One real clip already published on this site, shown as a format and rig example: a head-mounted stereo capture with hands in frame. It does not show a clinical procedure, a phantom or a mannequin.
No clinical procedure clip is published on the site, so this is labelled as a format and rig example. Request a sample to see delivered files. See stereo sample page, stereo + wrist-cam example videos, request an egocentric data sample.
FAQ
Sources
Tell us which of dressings, injections and checks you need, where each episode starts and ends, what phantom or mannequin stands in for the patient, the streams you want in the file and whether MCAP sits alone or beside a training format. We will quote reach, timeline and price against a written spec.