Now recruiting capture teams worldwide. Get paid to record first-person video of real work.

Apply to capture

Instrument sets

Stereo Surgical Instrument Handling Data

Handle, clean, sterilize and assemble a surgical instrument set: sterile processing work is many small items, two hands and a run of steps that hide the work from a single camera. This page covers the stereo and container side of instrument set assembly robot data for the one catalog row that names this work: what a calibrated stereo pair adds to stereo surgical instrument video, what MCAP is according to its own documentation, and what the site does not state. We run custom medical device data collection fast and on demand against a spec you write, as a sterile processing dataset delivered as MCAP, as custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Catalog row
"Surgical Instrument Technician" in Dental Lab & Medical Device Workshop, rated Hard
Status
Available on request, as the catalog lists it
Container
MCAP (.mcap), open source
License
Perpetual, buyer-owned

Why stereo and MCAP here

What do a stereo pair and an MCAP file add to instrument set video?

01

Depth on many small items

The site states that a stereo pair at human-eye spacing makes depth recoverable from every frame pair, so the instrument, the set and both hands sit in one frame. The site gives no depth accuracy figure for this row and states that metric depth on sub-centimetre parts is flagged rather than interpolated.

02

Fingers hidden by the instrument, flagged not guessed

The site states that fingers disappear behind objects and that the hands occlude the work. 3D hand pose carries a per-joint visibility flag, so a finger folded behind an instrument is marked as hidden rather than silently filled in.

03

Four verbs in one row

The row bundles handling, cleaning, sterilizing and assembling. Action segments carry start_ns, end_ns, verb and noun, so each step can be its own labeled span. Which verbs and nouns are used is set in the skill spec.

04

Several streams in one container

MCAP stores multiple channels of timestamped, pre-serialized data, so a stereo pair, hand pose and IMU can share one file on one log-time axis. Which streams go in the file is set in the skill spec.

Definition

What is stereo surgical instrument handling data?

It is first-person video of a person doing surgical instrument set work, recorded with two calibrated cameras, with hand pose and sensor streams alongside. People reach it through several phrasings: stereo surgical instrument video, instrument set assembly robot data, a sterile processing dataset, or custom medical device data collection at scale. Stereo describes the capture, and MCAP describes the container. They are separate choices.

The site does not state that stereo is already running on this row. The catalog lists a head-mounted phone or a multiview rig as the capture setup, so a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The data is human capture by paid, consented contributors, not robot or teleoperation data, so using it for a specific robot still needs your own retargeting step. Firsthand is not affiliated with Foxglove or the MCAP maintainers.

Catalog row

Which catalog row does this page rely on, and what is its status?

One, listed in lib/commercial-catalog-data.ts under the Dental Lab & Medical Device Workshop environment in the Healthcare & Life Sciences sector, whose examples line reads "Dental laboratory, sterile processing, medical device assembly." "Surgical Instrument Technician" is described as "Handling, cleaning, sterilizing and assembling surgical instrument sets." It is rated Hard, it lists "Phone (head-mounted) / multiview rig" as the capture setup, and it is Available on request.

Available on request means the work is scoped and recruited to spec when a buyer asks for it, rather than running as an open capture stream today. The site states no delivered hours and no delivered stereo footage for this row, so this page claims none. The environment examples name sterile processing, but the catalog has no separate sterile processing row. The catalog does not break this row into sub-steps and names no tool, so none is named here beyond what the description says.

Difference

How is this different from the egocentric surgical instrument handling page, the stereo dental lab page and the stereo veterinary clinic page?

This page is the stereo and MCAP treatment of a single Available on request row, Surgical Instrument Technician. The egocentric surgical instrument handling page covers six Available on request rows across Dental Clinic, Dental Lab & Medical Device Workshop, Wet Lab & Clinical Bench and Veterinary Clinic, including this one, and centres on tray setup, sterile packs and instrument pass as first-person video that is not stereo as its subject.

The stereo dental lab and instrument reprocessing page covers two rows in this environment, Dental Prosthesis Manufacturing and Surgical Instrument Technician, so the instrument row shares that page with dental prosthesis fabrication. The stereo veterinary clinic page covers five Available on request Veterinary Clinic rows, where one row, Surgical instrument setup, opens sterile packs and lays out instruments on a tray and the subject is animal care. This page is the one place where the instrument set row is the whole subject. The first table below reproduces the row, and the second sets the pages side by side.

Capture setup

What capture setup does the row list, and where does stereo come in?

The catalog lists "Phone (head-mounted) / multiview rig" for this row. The site does not state a stereo pair for it. Stereo applies through the site’s general stereo offer: a stereo pair is a capture choice agreed in the skill spec and confirmed on a Spec Pilot. The catalog does not say which of its two modes applies, and the site lists capture at a customer’s own facility as not offered, so the setting is agreed in the spec.

The site describes two stereo setups and does not say which one an order for this row uses. The hard-environments stack is a global-shutter 4K/60 head camera with a 120° lens, two 1080p/60 wrist cameras, a depth sensor at 848×480 and 30 fps, and 200 Hz 6-DoF IMUs. The off-the-shelf stereo head rig, dataset 06, is dual fisheye cameras at 1080p per eye, a 62 mm baseline, per-unit calibration, a 9-axis IMU and hand-tracking output. Camera count, resolution and frame rate for this row are Not stated.

Why stereo

Why capture surgical instrument set work in stereo?

A set is many items, and the work on it is done with two hands: one steadies an item while the other works on it, and the instruments and the fingers occupy a small part of the frame together. The catalog calls the row Hard. A stereo pair recovers depth from the calibrated frame pair rather than from a standalone depth sensor, so the geometry of the instruments and the hands sits in the same capture that shows the fingers working.

A stereo pair adds depth, not a verdict on whether a set is complete or correct. The site lists no completeness, quality or success label for this row, and none is claimed. The catalog also names no instrument types, set contents or equipment, so what counts as a finished set is Not stated and is written into the spec as accept criteria before capture.

Four verbs

What changes between handling, cleaning, sterilizing and assembling?

The row is a run of steps over a set of instruments: handle, clean, sterilize and assemble. A spec that says "process the set" gives one long action segment. A spec that names handling, cleaning, sterilizing and assembling as separate steps gives a sequence you can slice by step. Whether each of those is its own segment, and whether one instrument, one tray or one whole set is one episode, is a spec decision, not a default. The site does not set an episode length or a step count.

This page describes capture only. It gives no cleaning or sterilization procedure, no cycle parameters and no standard, because the site states none, and it is not guidance for infection control or safety. Instrument types, set contents, cleaning agents, sterilizers and washers are Not stated. The catalog names no inspection step for this row either, and no row elsewhere lists one, so none is claimed.

Occlusion

What do the hands and the instruments do to visibility?

At a bench the hands sit between the camera and the work, and a small instrument can vanish behind a finger. The site states that the hands occlude the work and that fingers disappear behind objects. That is why each hand joint carries a visibility flag, why contact events mark when the hand or an item meets a surface, and why instance masks are recorded per frame.

The site says two-handed fine motion and small objects are where occlusion bites, and it states that metric depth on sub-centimetre parts is flagged rather than interpolated. Whether an instrument component counts as such a part is Not stated, and so is a mask accuracy figure. The site also states that depth degrades on wet and reflective surfaces. Whether metal instruments or a wet bench count as reflective or wet is Not stated, and so is whether gloved hands change hand pose. A spec that needs a clear view of a step should name that step, ask for wrist views or a multiview angle, and confirm it on a Spec Pilot.

People and privacy

What does the site say about workers, consent, privacy and patient data?

The site states a consent chain for contributors. Every contributor signs a written release, each delivered episode links to it through consent artefact IDs, and a break at any of the six checkpoints discards the footage. An undocumented bystander is a stated reject criterion. For every delivered episode the site states that faces, plates and screens are detected and irreversibly blurred at ingest, that a reviewer confirms the result before anything ships, and that footage that cannot be de-identified is discarded. The anonymization record ships with the data.

This page does not imply that patients, clinical procedures or protected health information appear in capture, and the site states no patient-data handling for this environment. Whether paperwork or labels on a bench are blurred is Not stated. The catalog notes HIPAA compliance for its Hospital & Clinical Care environment, but states nothing like it for this one, so this page makes no regulatory, certification, sterilization or clinical claim. The site names no hospital, facility, city or region for this row and lists capture at a customer’s own facility as not offered. Who may be in frame is agreed in the spec. This is not legal advice, and the executed agreement controls.

Not stated

What does the site not state about instruments, equipment, standards and results?

Instrument types and set contents, equipment brands and models, cleaning agents, sterilizers and washers, sterilization standards and cycle parameters, regulatory or certification status, cycle times, error rates, facility types and sizes, customer and hospital names, and what a correct or finished result means are Not stated. They are set in the written spec, and this page claims none of them.

MCAP

What is MCAP, per its own documentation?

MCAP is an open source container file format for multimodal log data, supporting multiple channels of timestamped pre-serialized data. It is serialization-agnostic, so messages can be Protobuf, DDS (CDR), ROS, JSON and more. Its design is row-oriented and append-only, it stores schemas alongside the data, and it offers an optional index plus optional LZ4 or Zstandard chunk compression.

Reader and writer libraries exist for C++, Go, Python, Rust, Swift and TypeScript, and the mcap CLI can inspect, merge and split files. The foxglove/mcap repository is MIT-licensed; that is the MCAP project licence, not the licence of a Firsthand delivery. The MCAP hub page and the glossary entry cover the format in full.

Streams

Which streams can ship in an MCAP delivery of instrument set data?

Firsthand offers MCAP delivery, and dataset 06 is stated as shipping as MCAP. For this row the site states hand pose, a stereo-derived depth stream with confidence maps, IMU, contact events, action segments, per-frame instance masks and one task instruction per episode elsewhere. What it does not state is which of those are channels in an MCAP file, how they are named, what encoding they use or at what rate.

No topic name, message schema or rate is stated on this page, and none should be assumed. Name the streams that matter in the skill spec and confirm them on a Spec Pilot batch. The fourth table below sets each stream beside what the site states about it.

Failure modes

What goes wrong when you record instrument set work in stereo, and what does QA reject?

Hand occlusion by the instrument and the set is the first risk, under the visibility flag above. Two-handed fine motion with small items is the second, and how often it leaves the frame is Not stated. Calibration is measured at the start and end of every session under a 0.28 px RMS gate, and an episode is rejected for a sync breach beyond the ceiling for more than three consecutive frames, for the manipulated object out of frame for more than 15 percent of the episode, for an undocumented bystander or for a mismatch against the named skill spec.

A reject log ships with every batch, and rejected items are recaptured at the provider’s cost. The fifth table below lists the failure modes beside what the site states.

Scale

How do off-the-shelf coverage and custom collection combine for instrument set work?

The October 2026 off-the-shelf catalog lists ~324,800 hours across 18 datasets, of which stereo is ~64,800 hours across 9 datasets. None of them names surgical instrument handling, cleaning, sterilizing or set assembly, so for stereo capture of this task this is custom collection, not an off-the-shelf purchase.

It is custom collection at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs. Where coverage is deep, the median time from signed spec to first validated batch is a median of 48 hours; where it is thin, turnaround runs three to four weeks. The site does not say which applies to this row, which is Available on request, so the timeline is quoted against your spec. Collections are billed per validated hour, and rejected episodes and recaptures are never billed.

Ordering

How do you order a stereo surgical instrument handling dataset?

Describe the task and where each episode starts and ends: one handling, cleaning, sterilizing or assembling step on an instrument set, or the whole run. Say whether an episode covers one step, one instrument, one tray or a whole set, what counts as the object and what counts as a finished step, the setting and lighting, the privacy rule for the bench and for anyone in frame, and name the variation your pipeline is missing. Then name the streams you want in the file, the topic naming and message encoding your reader expects, whether calibration ships inside the file or beside it, and whether MCAP is the only delivery or sits beside RLDS, LeRobot or WebDataset.

Most programs begin with a Spec Pilot, a small first batch that proves the spec and accept criteria on real footage before you scale. The sixth table below shows what each spec element decides.

At a glance

What you get.

Catalog row
"Surgical Instrument Technician" (Dental Lab & Medical Device Workshop, Healthcare & Life Sciences); "Handling, cleaning, sterilizing and assembling surgical instrument sets."; Hard; Available on request
Catalog capture setup
Phone (head-mounted) / multiview rig. The site states no stereo pair for this row, so a stereo pair is agreed in the skill spec through the general stereo offer
Stereo head rig (06)
Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAP
Hand pose
21 joints per hand with a per-joint visibility flag
Sync
Hardware-triggered onto one clock; 2.0 ms cross-stream ceiling
Instrument types, set contents, cleaning agents, sterilizers, washers, hospital and facility names
Not stated; agreed in the skill spec
Regulatory or certification status, patient data
Not stated; nothing here is clinical, sterilization, infection-control or safety guidance
MCAP topics, schemas, rates
Not stated on the site; agreed in the skill spec
License
Perpetual, irrevocable, buyer-owned, non-exclusive by default

Every collection runs through the same seven-stage end-to-end custom collection pipeline. End-to-end custom data collection is a managed service that takes an AI data need from problem to owned dataset in a single accountable pipeline.

Catalog row

Which catalog row covers surgical instrument set work, and what is its status?

One row from lib/commercial-catalog-data.ts, reproduced as the catalog states it, plus items the catalog does not list.
Task (as listed)Description (as listed)EnvironmentSkillCapture setupStatus
Surgical Instrument TechnicianHandling, cleaning, sterilizing and assembling surgical instrument sets.Dental Lab & Medical Device WorkshopHardPhone (head-mounted) / multiview rigAvailable on request
Sterile processing task rowNamed in the environment examples; no separate task row is statedNot statedNot statedNot statedNot stated
Separate handling, cleaning, sterilizing or assembling rowsNot stated as separate rowsNot statedNot statedNot statedNot stated
Named instruments, sterilizers or washersNot statedNot statedNot statedNot statedNot stated

Compared pages

How does this page differ from the egocentric surgical instrument handling page, the stereo dental lab page and the stereo veterinary clinic page?

Each page and what it covers. Statuses are as the site lists them; none of these pages is claimed to hold delivered hours here.
PageWhat it coversRow status as listedRelation to this page
This page: stereo surgical instrument handling dataOne row: Surgical Instrument TechnicianAvailable on requestStereo pair and MCAP delivery for the instrument set row alone
Egocentric surgical instrument handling dataSix rows across Dental Clinic, Dental Lab & Medical Device Workshop, Wet Lab & Clinical Bench and Veterinary Clinic, centred on tray setup, sterile packs and instrument passAll six Available on requestNot stereo as its subject; includes Surgical Instrument Technician and five rows this page does not cover
Stereo dental lab and instrument reprocessing dataTwo rows: Dental Prosthesis Manufacturing and Surgical Instrument TechnicianBoth Available on requestStereo and MCAP treatment that shares this row with dental prosthesis fabrication
Egocentric dental lab and medical device workshop dataThe same two rows as the stereo dental lab pageBoth Available on requestNot stereo as its subject; the first-person page for the same two rows
Stereo veterinary clinic dataFive Veterinary Clinic rows, including Surgical instrument setup, with stereo and MCAPAll five Available on requestAnother stereo clinic page; animal care, not instrument set processing
Egocentric stereo data in MCAPThe stereo and MCAP hub across tasksNot applicableGeneral stereo and MCAP treatment; not specific to instrument sets

Stated vs not stated

What the site states about stereo capture for surgical instrument set work

Left to right: the item, what the site states, and what that means for an order. “Not stated” means the site does not say.
ItemWhat the site statesFor an instrument set order
Catalog capture setupPhone (head-mounted) / multiview rig on the rowNo stereo pair is stated for this row; a stereo pair is a spec choice through the general stereo offer
Stereo head rig (dataset 06)Dual fisheye, 1080p per eye, 62 mm baseline, per-unit calibration, 9-axis IMU, hand-tracking output; shipped as MCAPWhether an order uses it: Not stated
Hard-environments stack4K/60 head camera, 120° lens; two 1080p/60 wrist cameras; depth 848×480 at 30 fps; 200 Hz 6-DoF IMUWhether an order uses it: Not stated
Hands, contacts and masksPer-joint visibility flag on hand pose; contact events; per-frame instance masks stored at 848×480Stated; what counts as the object is set in the spec
Hand occlusionThe hands occlude the work and fingers disappear behind objects; visibility flags and instance masks address itStated; how often an instrument leaves the view is Not stated
Depth on sub-centimetre partsMetric depth on sub-centimetre parts is flagged rather than interpolatedWhether an instrument component counts: Not stated
Depth on wet and reflective surfacesDepth degrades on wet and reflective surfacesStated; behaviour on metal instruments and a wet bench is Not stated
Gloved handsNot statedSettle in the spec and on a Spec Pilot
CalibrationMeasured every session and shipped with the episode; 0.28 px RMS reprojection gateStated
SynchronizationHardware-triggered onto one clock; 2.0 ms cross-stream ceilingStated
Row statusAvailable on requestNo volume, hours or delivered stereo footage is claimed on this page
Face blurring and anonymizationFaces, plates and screens detected and irreversibly blurred at ingest; reviewer confirms; footage that cannot be de-identified is discardedStated; record ships with the data
Patients, patient data, regulatory and certification statusNot stated for this environmentNot claimed; none is implied
Instruments, equipment, standards, cycle parametersNot statedNot claimed; set in the spec
Cycle time, error rate, result qualityNot statedNot claimed on this page
MCAP topics, schemas, ratesNot statedAgreed in the skill spec

Stream map

Streams for surgical instrument set data and what the site states about an MCAP file

Left to right: the stream or item, what the site states about it, and what the site states about it in MCAP. “Not stated” means the site does not say.
Stream or itemWhat the site statesIn an MCAP delivery, as the site states it
Stereo pairDual fisheye, 1080p per eye, 62 mm baseline (dataset 06)Dataset 06 is stated as shipped as MCAP; channel layout not stated
Depth with confidence mapsRecovered from the calibrated stereo pair; confidence map masks low-confidence regionsNot stated
Hand pose21 joints per hand with a per-joint visibility flagChannel layout and joint order not stated
IMU9-axis (dataset 06); 6-DoF at 200 Hz (hard-environments stack)Channel layout and rate not stated
Contact events, action segments, instance masksListed in the site episode schema; masks stored at 848×480Not stated
Task instructionOne per episodeNot stated
CalibrationMeasured every session and shipped with the episodePlacement in the file not stated
Topic names, schemas, encoding, ratesNot statedAgreed in the skill spec

Failure modes and QA

What goes wrong in instrument set capture, and what QA does about it

Left to right: the failure mode, what the site states, and what a buyer sees. “Not stated” means the site does not say.
Failure modeWhat the site statesWhat a buyer sees
Fingers hidden behind an instrumentPer-joint visibility flag on hand pose; wrist views and a multiview angle available in the specA hidden joint flagged as hidden, not guessed
Instrument or set out of framePer-frame instance masks; episode rejected when the object is out of frame for more than 15 percent of itReject log per batch; what the object is gets set in the spec
Two-handed fine motion with small itemsThe catalog rates the row Hard; no frequency or error rate is statedSegment boundaries and contact events set in the spec
Reflective metal instruments or wet surfacesDepth degrades on wet and reflective surfaces; which bench items count is Not statedConfidence maps ship with depth; Spec Pilot settles it
Gloved handsNot statedSettle in the spec and on a Spec Pilot
Invalid depth in the manipulation regionEpisode rejected when more than 35 percent of depth pixels are invalidReject log per batch
Bystander or paperwork in frameFaces, plates and screens blurred and reviewer-confirmed; undocumented bystander rejected; footage that cannot be de-identified is discarded; blurring of paperwork Not statedReject log per batch; anonymization record ships with the data
Calibration driftSolved at the start and end of every session; 0.28 px RMS gateCalibration shipped with the episode
Sync breachRejected beyond the 2.0 ms ceiling for more than three consecutive framesReject log per batch
Spec mismatchRejected for a mismatch against the named skill specReject log per batch

Skill spec

What the skill spec fixes for a stereo surgical instrument handling order

Spec element and where a buyer can see it in the delivery. “Set per program” means the site states no fixed value.
Spec elementWhat it decidesWhere it shows up
RowSurgical Instrument Technician, worded as the catalog words itSet per program; status as listed
Steps and orderWhat one step is: handling, cleaning, sterilizing or assembling, and in what orderEpisode boundaries and action segments; set per program
Episode scopeOne step, one instrument, one tray or a whole setSet per program; the site does not state an episode length
Object definitionWhether the instrument, the tray, the set or a tool counts as the objectContact events and the out-of-frame reject rule; set per program
Rig and streamsStereo pair, multiview angles, wrist views, hand pose, IMU and any othersThe channels you can read; confirm on the Spec Pilot
Setting and lightingWhere capture runs and how the bench is litSet per program; the site states no lighting setup for this row
MCAP topics and encodingWhat your reader expectsSet per program
Delivery formatHow the data is packagedMCAP alone, or beside RLDS, LeRobot, WebDataset, HDF5 or zarr
Accept criteriaWhat counts as a completed step or a finished set, and what passes QAReject log per batch
PrivacyHow anyone, faces, screens and paperwork in frame are handledFaces, plates and screens blurred at ingest; other items agreed in the spec
LicenseWho owns the datasetPerpetual, irrevocable, buyer-owned, non-exclusive by default

Example footage

What does egocentric stereo capture look like?

One real clip already published on this site, shown as a format and rig example: a head-mounted stereo capture with hands in frame. It does not show surgical instrument handling, cleaning, sterilizing or assembling.

egocentric_stereo_sample.mp4 · head-mounted stereo · hands in frame · 1920 × 1080 web preview · rig example, not an instrument set clip

No surgical instrument clip is published on the site, so this is labelled as a format and rig example. Request a sample to see delivered files. See stereo sample page, stereo + wrist-cam example videos, request an egocentric data sample.

FAQ

Stereo surgical instrument video, answered.

What is stereo surgical instrument handling data?
First-person video from two calibrated cameras of a person handling, cleaning, sterilizing and assembling surgical instrument sets, with depth and hand pose alongside.
Which catalog row covers it, and what is its status?
One, under Dental Lab & Medical Device Workshop: "Surgical Instrument Technician" (handling, cleaning, sterilizing and assembling surgical instrument sets). It is Hard and Available on request. This page claims no delivered hours.
Is stereo already captured on this row?
Not stated. The catalog lists a head-mounted phone with a multiview rig for the row. A stereo pair is agreed in the skill spec through the general stereo offer and confirmed on a Spec Pilot.
How is this different from the egocentric surgical instrument handling page?
That page covers six instrument handling rows across four environments as first-person video that is not stereo as its subject. This page is stereo capture of the one Surgical Instrument Technician row, with MCAP delivery.
How is it different from the stereo dental lab page and the stereo veterinary clinic page?
The stereo dental lab page covers this row together with dental prosthesis fabrication. The stereo veterinary clinic page covers five Veterinary Clinic rows where the subject is animal care. This page covers the instrument set row alone.
Is there an off-the-shelf surgical instrument dataset?
No listed dataset names this task, so stereo capture of it is custom collection, at scale through a network of thousands of capture providers, for a consistent, recurring data flow to labs.
Does the data include patient information or sterilization guidance?
No patient data is implied, and the site states no patient-data handling, regulatory status or sterilization standard for this row. Faces, plates and screens are irreversibly blurred at ingest and a reviewer confirms the result. This page gives no clinical or safety instructions.
Can it be delivered as MCAP, and what is in the file?
Yes. Firsthand offers MCAP delivery. Topic names, message schemas and rates are Not stated on the site and are agreed in the skill spec.

Sources

Where is MCAP documented?

Request a custom surgical instrument set dataset delivered as MCAP.

Tell us the task and where each episode starts and ends, whether an episode covers one step, one instrument, one tray or a whole set, what counts as the object and as a finished step, the setting and lighting, the privacy rule for the bench and for anyone in frame, the streams you want in the file and whether MCAP sits beside RLDS, LeRobot or WebDataset. We will quote reach, timeline and price against a written spec, with a first validated batch proven on a Spec Pilot.