Egocentric Pharmacy and Lab Bench Task Video Data

Egocentric pharmacy and lab bench video data is first-person video of a person doing dispensing or bench work, recorded from their own point of view with synchronized sensor streams alongside. The camera sits on the head, close to what a robot’s onboard camera would see at a counter or bench, so the frames show what the hands are doing to the container rather than a distant view of a room. The same idea sits behind any pharmacy robot training data request or lab bench video dataset, applied here as a custom collection scoped to your tasks.

On this site the data is human capture: a paid, consented contributor wears the rig and performs a task from a written skill spec. It is not robot or teleoperation data. This page differs from the pouring and liquid handling page, a general task family, and from the dental lab page.

The task catalog lists two relevant environments under Healthcare & Life Sciences. Pharmacy & Laboratory has seven rows, and its example line reads "Medication dispensing, liquid handling, pipetting, labeling, safety protocol adherence -- requires regulatory compliance". Wet Lab & Clinical Bench has fourteen rows, and its example line reads "Burette, pipettes, balance, labware, phantom for any invasive step".

The tables below list the seven pharmacy and laboratory rows and the fourteen bench rows as the catalog states them. All twenty-one are Available on request, so the work is scoped and recruited to spec when a buyer asks.

Counting pills is not its own row; the dispensing row says to count or measure the medication. The site states no equipment models, reagents, medications, pharmacy or lab types or sizes, or cycle times.

Each attempt is delivered as one episode. On this site the standard episode carries action segments, 3D hand pose for both hands with a per-joint visibility flag, contact events that mark when a hand meets an item, an objects list, and per-frame instance masks stored at the depth resolution of 848×480. The capture setup table below lists the stated streams and sync figures.

The catalog lists two capture modes for these rows. The seven pharmacy and laboratory rows use a head-mounted phone with a multiview rig. The fourteen bench rows use a multiview rig with handheld grippers. That second mode is not a head-mounted first-person view, so the spec states which mode a program needs. Force and tactile readings are not among the listed streams.

Action segments divide an episode into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For bench work, a segment might be one dispense, one tip change or one tube filled, as written in the skill spec.

Validation runs against signed accept criteria written into the spec before capture, and a reject log ships with each batch. Episodes are machine-checked against stated reject criteria, such as a sync breach or the manipulated object out of frame for more than 15 percent of the episode, and a human reviewer then checks each one. Only footage that clears the criteria is billed as a validated hour.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata, with readable converter source alongside. The license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity available as a separate term. The executed agreement controls.

Tubes, bottles and pipettes are small and are often hidden by the fingers that hold them. The catalog describes the work in exact terms: dispensing exact volumes into tubes and changing tips, filling tubes to a mark, pouring to a graduated cylinder mark, and titrating to a colour endpoint. That is why hand pose carries a per-joint visibility flag, why contact events mark when the hand meets the item, and why masks are recorded per frame. The challenge table below maps each difficulty to the stream or label this site states.

A robot foundation model paper on arXiv organizes its training data as a pyramid, with large quantities of web data and human videos at the base, synthetic data in the middle and real robot data at the top. The same paper says human videos have no explicit action labels, so it learns latent actions to label them. That is robot-model research, not Firsthand data, and Firsthand has no relationship with its authors.

The catalog describes the head-mounted phone as recording the first-person view of real work. The site also states that Firsthand does not capture at a customer’s own facility. Which pharmacy or laboratory setting a given program uses is not stated here and is agreed in the skill spec before capture begins.

Every contributor signs a written release, and each delivered episode links to it through consent artefact IDs. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers. A human reviewer confirms that no un-consented person is identifiable in any frame, and the anonymization record ships with the data.

The site’s privacy guide names printed text such as forms, badges and mail among recurring identifiers in egocentric footage. Whether prescription text or printed labels on a bench are blurred is not stated, and neither is any patient-data handling for these environments. The catalog notes HIPAA compliance for its Hospital & Clinical Care environment and a regulatory-compliance line for Pharmacy & Laboratory, but this page makes no regulatory, medication-safety, controlled-substance, biosafety or clinical claim. Who may be in frame is agreed in the spec.

From signed spec to first validated batch, the published timeline is a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. All twenty-one rows are Available on request, so the range that applies is confirmed when the spec is quoted rather than assumed.

Describe the bench or counter tasks and their order, the containers and tools involved, what counts as a finished step, and the failure or variation cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins. Our skill spec guide walks through how to write one.

Most programs begin with a Spec Pilot, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales. Collections are billed per validated hour. Counts of sites, episodes and hours are set in the skill spec.