Both hands in frame, not one
Coordinated two-handed work asymmetric grasps, one hand stabilizing while the other manipulates needs both hands tracked through the whole sequence, not a single dominant-hand crop.
Manipulation
Custom Bimanual Manipulation Data Collection
Two-hand coordination is the hard part of manipulation data. We capture it egocentrically, with both hands and the handoff between them recorded on one clock, against a spec you write.
- Streams
- Ten, synced under 2 ms
- First batch
- Median 48 h
- Billing
- Per validated hour
- License
- Buyer-owned
Requirements
What does bimanual manipulation data need?
Per-hand pose and contact, on one clock
3D hand pose and contact events for each hand, aligned to the same clock as the video and depth, so a policy can learn when and how the two hands hand off.
Occlusion marked, not guessed
When one hand occludes the other or the object, visibility flags mark it rather than letting a tracker hallucinate a pose.
Rights you can ship on
A perpetual, buyer-owned commercial license backed by a documented consent chain, so a policy trained on the data can go into a product.
Why it matters
Why is bimanual coordination hard to capture well?
Most manipulation data defaults to a single-arm, single-camera setup because it is simpler to rig. But many real tasks, from opening a jar to folding fabric, depend on what the second hand is doing at the same instant: stabilizing, supporting, or handing off.
Capturing that requires two hands tracked simultaneously with per-hand visibility, not a wide shot that loses hand detail, and not a single wrist camera that only sees one side of the interaction.
Capture setup
How is bimanual data captured egocentrically?
A head-mounted camera keeps both hands in frame from the viewpoint a two-handed policy will actually see from, entering frame from below through the reach-grasp-release cycle. Optional wrist cameras add close-range detail for fine manipulation on each hand.
3D hand pose is tracked per hand with visibility flags, and contact events mark exactly when and where each hand makes and breaks contact, all aligned to depth and video on one clock.
The spec
How does a custom bimanual manipulation spec work?
You describe the two-handed tasks your policy needs. We turn it into a written spec covering tasks, environments, regions, capture tier, annotation depth, and accept criteria, then quote reach, timeline, and price against that document before any capture begins.
The collection runs through the same seven-stage end-to-end custom collection pipeline as every Firsthand program: contributor recruiting, capture, per-batch QA, anonymization, and licensed delivery. Rejected items are recaptured at our cost.
Fast · on demand
How fast does a first batch ship?
Where coverage is already deep, the first validated batch lands in a median of 48 hours from signed spec. Thin environments typically take three to four weeks, because vetting operators there is the bottleneck, and we quote that timeline up front.
On-demand top-ups run against the same spec and accept criteria after each training run, so new bimanual data stays consistent with what you already hold.
Rights
What do licensing and consent look like?
Every custom collection ships under a perpetual, buyer-owned commercial license, and exclusivity is available so the data is never resold. Each episode links to a signed contributor release through a documented consent chain.
Faces, plates, and screens are detected and irreversibly blurred at ingest, and a reviewer confirms the result before anything ships. The anonymization record is delivered with the data.
Bimanual need vs delivery
What does a bimanual manipulation policy get, need by need?
| Bimanual manipulation need | What Firsthand captures |
|---|---|
| Both hands in frame | Head-mounted camera keeping both hands visible through the full task cycle |
| Per-hand pose | 3D hand pose tracked separately for each hand, with visibility flags for occlusion |
| Per-hand contact | Contact events for each hand, aligned to the same clock as video and depth |
| Fine manipulation detail | Optional wrist cameras for close-range detail on each hand |
| Depth | Synchronized depth stream for scene and contact geometry |
| Action structure | Instance segmentation, action segments, grasp and contact events |
| Quality | Measured against signed accept criteria; billed per validated hour; reject log with every batch |
| Formats | RLDS, LeRobot, WebDataset, plus HDF5, zarr, and Rerun .rrd |
Every collection runs through the same seven-stage end-to-end custom collection pipeline.
FAQ
Custom bimanual manipulation data collection, answered.
- What is custom bimanual manipulation data collection?
- It is real-world, first-person demonstration data of two-handed tasks captured against a written spec: egocentric video with per-hand 3D pose, contact events, and depth for both hands, aligned on one clock. Firsthand handles contributors, capture, QA, consent, and delivery.
- Why not use a single-arm manipulation dataset instead?
- A single-arm setup only records one hand, so it cannot teach coordination, handoff, or stabilization between two hands. A custom bimanual collection tracks both hands with per-hand pose and contact, specified and licensed to you.
- Does capture include contact events for both hands?
- Yes. Each episode includes 3D hand pose and contact events tracked separately for each hand, aligned to the same clock as the video and depth streams.
- How is custom bimanual data billed?
- Per validated hour, quoted against the written spec before any capture. You pay only for data that clears the accept criteria, and rejected items are recaptured at our cost.
- Which delivery formats are supported?
- RLDS, LeRobot, and WebDataset, plus HDF5, zarr, and Rerun .rrd. Converter source is readable, so you can retarget the schema to your own loader.
- Can we start with a small pilot?
- Yes. A Spec Pilot is a small, fast first batch that proves the spec and accept criteria before you scale volume on demand.
Sources
Where are the delivery formats documented?
- RLDS (Reinforcement Learning Datasets) (opens in a new tab)Official repository for the RLDS episode format. github.com
- LeRobot (opens in a new tab)Official Hugging Face repository for the LeRobot dataset format and tooling. github.com
- LeRobot documentation (opens in a new tab)Hugging Face documentation for loading and using LeRobot datasets. huggingface.co
- WebDataset (opens in a new tab)Official repository for the WebDataset sharded format. github.com
Request a custom dataset.
Tell us the two-handed tasks your policy needs to perform. We will quote reach, timeline, and price against a written spec, with a first validated batch in a median of 48 hours where coverage is deep.