Egocentric Lab Pipetting and Sample Handling Data

Egocentric lab pipetting and sample handling data is first-person video of one person doing bench work, recorded from that person’s own point of view while they work. A paid, consented contributor wears the capture rig and performs the task written in a skill spec, and each attempt is delivered as an episode with synchronized streams, hand pose and labeled action segments. It is human capture. It is not robot or teleoperation data, and it is not a fixed corpus you download.

The site states the tasks in its commercial task catalog. In the Pharmacy & Laboratory environment, the row "Pipetting" reads: "Draw liquid into the pipette, dispense exact volumes into tubes and change tips." It is rated Medium, captured with a head-mounted phone or a multiview rig, and listed as available on request. The row "Lab protocols (pipetting, titration, sample handling)" reads: "Pipette reagents, titrate to endpoint and handle samples following the written protocol." It is rated Hard. In the Wet Lab & Clinical Bench environment, "Fill + cap sample tubes" reads "Fill tubes to a mark and cap," and "Cap, vortex + rack tubes" reads "Cap sample tubes, vortex-mix, and rack them in order." Both are rated Medium.

Neighbouring pages cover different subjects. Egocentric pharmacy and lab bench data spans medication dispensing, labeling and weighing as well as lab work, and egocentric pouring and liquid handling data follows liquid transfer out of the lab into kitchens, bars and farms. Healthcare egocentric data is the broad industry page. The subject here is the bench slice of that cluster: pipetting and sample-tube work first, with the other bench rows alongside.

People reach this topic through pipetting robot training data, lab bench video datasets, egocentric sample handling data, or custom lab automation data collection. They describe one request from different angles. The work is the same: write the task down, collect it on demand, and receive validated episodes.

The first table below reproduces fourteen catalog rows. Five come from Pharmacy & Laboratory: "Pipetting", "Liquid handling", "Lab protocols (pipetting, titration, sample handling)", "Laboratory Work" and "Labeling". Nine come from Wet Lab & Clinical Bench: "Cap, vortex + rack tubes", "Fill + cap sample tubes", "Rack + label tubes", "Load + balance centrifuge", "Pour + measure reagent", "Weigh reagent", "Prepare a slide", "Swab + streak plate" and "Titrate to endpoint".

Two groups matter most for this page. The liquid-transfer group is pipetting and liquid handling, where the catalog says to measure and transfer liquids between vessels with pipettes, cylinders and dispensers. The tube group is filling, capping, vortexing, racking and labeling, and loading tubes balanced in a centrifuge. The remaining rows prepare a slide, swab a sample and streak a plate, weigh a reagent and titrate to a colour endpoint.

Some rows are left out on purpose. The catalog also lists medication dispensing and safety protocol adherence under Pharmacy & Laboratory, and clinical rows such as a mannequin cuff, a syringe connection, an instrument pass, sterile field setup and a phantom dressing under Wet Lab & Clinical Bench. They are about pharmacy or clinical work rather than sample handling, and other pages cover them.

The site does not state a pipette type, tip type, tube or vial type, volume, assay or protocol. The word "plate" appears only in "Swab + streak plate", and the catalog lists no well-plate, microplate or vial-rack row. If you need one of those, say so in the request and settle it in a Spec Pilot.

Each attempt is delivered as one episode, and bench work needs a decision up front about what one attempt is. It could be one transfer, one tube, one rack or a whole protocol run. The site does not set an episode length or the number of episodes, so the skill spec does. Inside the episode, the synchronized streams run from start to end, and action segments divide it into labeled spans.

The choice matters because of two machine-checked rules. An episode is rejected if the manipulated object is out of frame for more than 15 percent of it, and if more than 35 percent of depth pixels in the manipulation region are invalid. At a bench the pipette, the tip and the tube are small and close to the hands, so the spec should say which one counts as the manipulated object and where the manipulation region is at each step.

The site states that depth degrades on wet and reflective surfaces, and that a confidence map is delivered so low-confidence regions can be masked. It does not say how glass labware, liquid in a vessel or a transparent tube is handled. If your policy depends on any of those, raise it at the start and settle it in a Spec Pilot rather than assuming it.

Inside each episode, action segments divide the recording into start- and end-timestamped spans, each carrying start_ns, end_ns, verb and noun. For this task family a segment can be one step as written in the skill spec, so a pipetting run becomes a sequence of labeled steps such as drawing, dispensing and changing the tip, rather than one long clip. The other standard fields are hand_joints and hand_visible for 3D hand pose, contact_events for the moment the hand meets an object, objects, and instance_masks, stored at the depth resolution of 848×480 so they line up with the depth stream.

Hand pose is delivered for both hands with a per-joint visibility flag for occluded fingers. That is the label that carries how the fingers hold the pipette, press the plunger and steady the tube. The site describes contact events as the moment the hand meets an item, and does not say whether a pipette tip or a tube cap counts as the item, so the spec settles what a contact event means for your task.

The site lists no label for the liquid or the result. There is no stated volume, liquid level, tip-attached state, sample identity or pass-or-fail score, and no contact force or tactile reading. Camera pose, a head trajectory and a map of the workspace are likewise not among the listed deliverables. The site states 6-DoF IMU data at 200 Hz from head- and wrist-mounted sensors, and nothing more about motion.

Delivery is in the format your stack already reads: RLDS, LeRobot, WebDataset, HDF5, zarr or Rerun .rrd, or raw streams with metadata. Readable converter source ships alongside the data so the schema can be retargeted to your own loader. The official documentation for three of these formats is linked in the sources below. Collections are billed per validated hour, and the license is perpetual, irrevocable and worldwide, buyer-owned and non-exclusive by default, with exclusivity available as a separate term.

A published robot foundation model paper, GR00T N1, linked in the sources below, describes a data pyramid with web data and human video at the base, synthetic data in the middle and real robot trajectories at the top. First-person human video of hand work sits in that base layer. Human capture does not replace robot data, and this page does not claim that it does. The glossary entry on the data pyramid explains how the layers trade volume for embodiment closeness.

Bench work puts a lot into the fingers. On that point, the site’s tool use page says that grip changes and tool hand-off between hands live in finger-level detail that RGB video alone cannot resolve, and that each needs 3D hand pose tracked through reach, grasp and tool manipulation. That page is about driving and fastening tools, not pipettes, so this page does not claim the same findings for lab work. The same page says metric depth on sub-centimetre parts is flagged rather than interpolated. Whether a pipette tip or a tube counts as such a part is not stated, and neither is a mask accuracy figure for either.

What carries over is the reasoning. A clip that shows a bench somewhere in frame does not carry what a policy needs. The hands, the pipette, the tube and the step boundaries have to be in the episode, and a spec that names them lets each step become one labeled segment. A custom collection lets you aim at the step your model is missing, such as changing a tip or capping a row of tubes in order.

This page does not claim that any public corpus lacks lab data, and it does not claim a training outcome, a success rate or an accuracy gain from any dataset. The tradeoff to plan for is the embodiment gap between a human hand holding a pipette and a robot platform doing the same, which retargeting or pairing with robot data has to close.

The site describes two capture setups. The Pharmacy & Laboratory rows use a head-mounted phone or a multiview rig. The site describes that setup as a head-mounted phone recording the first-person view of real work on site, with a multiview rig adding synchronized third-person angles when the spec asks for them. The Wet Lab & Clinical Bench rows use a multiview rig with handheld grippers, which the site describes as a staged working set. The site does not explain the grippers further.

The site names no lab, hospital, university, institution or city, and it lists capture at your own facility as not offered. This page therefore makes no claim that episodes are recorded in a particular lab, and none that they are recorded at yours. Capture is run to your skill spec using the settings and contributors Firsthand arranges.

Every contributor signs a written release before capture, and each delivered episode links to that record through consent artefact IDs in the batch data card. The consent chain has six checkpoints, from before capture to after delivery, and a break at any one discards the footage rather than shipping it at a lower grade. At ingest, automated detection finds faces, plates and screens and blurs them irreversibly, and speech is scrubbed of names, addresses and card numbers while ambient audio is kept. A human reviewer confirms that no un-consented person is identifiable in any frame, and footage that cannot be de-identified is discarded. An undocumented bystander is also one of the machine-checked reject criteria. The anonymization record is delivered with the data.

Tube labels, sample labels and container markings are not listed as blur categories on the site, so this page does not claim they are blurred. Labeling and racking put printed text in close view by nature, so if label text in frame matters for your use, raise it in the spec so the handling is agreed before capture rather than assumed afterwards. The catalog notes that the Pharmacy & Laboratory work requires regulatory compliance, and this page makes no regulatory, biosafety, contamination-control, clinical or diagnostic claim of its own. The site states no patient-data handling for these environments, and this page covers no patient-facing task. None of this is legal advice, and the executed agreement controls.

From signed spec to first validated batch, the site states a median of 48 hours where coverage is deep and three to four weeks where it is thin and contributors need to be recruited first. The site does not say whether the laboratory environments count as deep or thin, so ask which timeline applies when you request the quote. Speed comes from a warm pipeline, not from cutting consent, review or anonymization.

The site also states program milestones: a pilot reaches its first delivery in two to three weeks, and a standing program reaches full rate in four to six weeks. After the first batch, on-demand top-ups run against the same accept criteria, so you can add steps, vessels or protocols without rewriting the spec. The site does not state a turnaround for those later batches, so plan them in your quote.

Only footage that clears the signed accept criteria is billed as a validated hour, and rejected items are recaptured at our cost. The site does not publish a price list, a minimum order or a batch size for this task family, so those are set in your quote. Collection is fast and on demand in the sense that you do not need a standing program: a focused brief is a normal starting point.

Where this site does not state something, this page says "Not stated" rather than filling the gap. There is no listed label for volume, liquid level, sample identity, assay or result quality, no instrument or consumable specification beyond the pipettes, tubes, plate, slide and balance the catalog rows name, no task duration, no site count, no region or city, no episode count and no success or quality label. Nothing here reports a robot result, a benchmark or a model improvement.

Counts of contributors, hours and tasks are set in your skill spec. A task outside the rows quoted above is settled in a Spec Pilot rather than assumed.

Describe the step, the vessel, the setting and the variation or failure cases your pipeline is missing. That becomes a written skill spec, quoted for reach, timeline and price before any capture begins, and most programs run a Spec Pilot first, a small, fast first batch that proves the spec and the accept criteria on real footage before volume scales. A published sample recording is on the sample data outputs page if you want to see the streams before you write a spec.

Three choices matter most for this family. Name the step and the item rather than a broad activity such as "lab work", because that is what lets each step become one action segment. Decide which environment you want, since the two use different capture setups. And name where the episode starts and stops: one transfer, one tube, one rack or a whole protocol run, since the site does not set an episode length.