Glossary
What Is Dexterous Manipulation?
Last reviewed: September 2026
Short answer
Dexterous manipulation is fine-grained, multi-finger manipulation — regrasping, in-hand reorientation, precision pinches, tool use — as distinct from a single wrist or parallel-gripper motion. Learning it needs finger-level 3D hand pose and contact events tracked through occlusion, not just a wrist position, captured fast and on demand against a written spec.
01
Why it matters
Most manipulation data defaults to a wrist-level pose or a parallel-gripper state because it is simpler to log. But fine manipulation happens at the fingertip, and that signal is lost the moment capture drops to wrist level, leaving a policy unable to learn grasp and release geometry it never saw.
02
How Firsthand handles it
Firsthand captures dexterous manipulation egocentrically from a head-mounted camera, with 3D hand pose tracked through the reach-grasp-adjust cycle, visibility flags marking occlusion, and contact events aligned to the same clock as video and depth. An optional wrist camera adds close-range detail. Custom collection runs against a written spec, with a median 48-hour first batch where coverage is deep, billed per validated hour under a buyer-owned license.
03
Dexterous manipulation training needs and what Firsthand delivers for each.
| Need | What Firsthand captures |
|---|---|
| Hand and object in frame | Head-mounted camera keeping the hand and object visible through the full task cycle |
| Finger-level pose | 3D hand pose tracked through the sequence, with visibility flags for occlusion |
| Contact detail | Contact events aligned to the same clock as video and depth |
| Fine manipulation detail | Optional wrist camera for close-range detail on the hand |
| Depth | Synchronized depth stream for scene and contact geometry |
| Formats | RLDS, LeRobot, WebDataset, plus HDF5, zarr, and Rerun .rrd |
05
Frequently asked questions
- What is dexterous manipulation?
- Dexterous manipulation is fine-grained, multi-finger manipulation — regrasping, in-hand reorientation, precision pinches, tool use — as opposed to a single wrist or parallel-gripper motion.
- Why is it harder to capture than general manipulation?
- General manipulation data can log a wrist or gripper position. Dexterous manipulation needs 3D hand pose tracked through the reach-grasp-adjust cycle with visibility flags, so finger-level detail is captured rather than lost.
- Does dexterous manipulation data include contact events?
- Yes. Firsthand aligns contact events to the same clock as the video and depth streams, marking exactly when and where the hand makes and breaks contact.
- How is occlusion of the fingers handled?
- Visibility flags mark occluded joints per frame rather than letting a tracker guess a hidden pose, so the policy learns from what was actually observed.
- How fast can custom dexterous manipulation data be collected?
- Where coverage is already deep, the first validated batch lands in a median of 48 hours from a signed spec. Thin environments typically take three to four weeks.