What Is DexCap?

DexCap is introduced in the paper DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation (arXiv 2403.07788), with an official project page and GitHub repository. The authors name two challenges: existing hand motion capture systems are hard to carry, and translating mocap data into effective robot policies is difficult. DexCap is their portable system and DexIL is their algorithm for learning from it.

DexCap is third-party research. Firsthand is not affiliated with its authors or their institutions, and makes no claim that DexCap uses Firsthand data or hardware or that any model was trained on Firsthand data. The site does not state that Firsthand collects with electromagnetic gloves or a chest camera rack. It states custom egocentric collection to a written spec, fast and on demand, with 3D hand pose at 21 joints per hand, calibrated stereo depth, 6-DoF IMUs at 200 Hz and a separate tactile glove tier.

Per the project page, a chest camera rack carries an RGB-D LiDAR camera and three SLAM tracking cameras, and a backpack mini-PC and power bank run the system for approximately 40 minutes. The tracking cameras are moved to hand mounts to track the palms, while motion capture gloves measure finger positions relative to the palm using an electromagnetic field (EMF).

Per the repository, each recorded frame holds a chest-camera color and depth image, the chest camera 6-DoF pose, both hands 6-DoF poses and 3D hand joint positions in the palm frame.

For observations, RGB-D frames become 3D point clouds in robot operation space, and a quick-release buckle swaps the same camera between human and robot in less than 20 seconds. For actions, fingertip inverse kinematics maps the human fingers to the 16-dimensional joint positions of a robot hand.

A Diffusion Policy takes the point cloud as input and outputs a sequence of future goal positions; the repository states the default configuration generates 20 steps of actions for both hands and arms, 46 dimensions in total. DexCap also supports optional human-in-the-loop correction for fine-tuning.

The project page says DexCap is about three times faster than teleoperation in data collection throughput. That is the authors’ claim, and Firsthand draws no throughput comparison from it. The authors report results on their own robot, including a bimanual policy learned from 30 minutes of human mocap data.

DexCap puts instrumented gloves on the demonstrator hands. In the Universal Manipulation Interface, a handheld gripper carries the camera, so every recorded motion is a gripper motion. Neither is video-only egocentric capture, and this page makes no ranking between them.

It is one worked design for dexterous data: wrist and finger motion plus 3D scene observations, collected with a portable rig instead of a robot, and evaluated across six dexterous manipulation tasks. The data is tied to the rig: point clouds in robot operation space, glove joint positions and retargeting to a specific robot hand.

The repository releases collection, processing, dataset-building and training code under the MIT License. Dataset licence terms are Not stated in the sources read.

When a dexterous program needs tasks and environments an existing capture rig does not cover, specified in writing. Firsthand does not build or train models, and makes no claim that its capture replaces mocap glove data. If contact signals matter, ask for the separate pilot-ready tactile glove tier, time-synced to a head-mounted stereo camera by hardware timestamps and an IMU.

Confirm the spec on a Spec Pilot, a small fast first batch. Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is DexCap?

A portable hand motion capture system from a 2024 paper, with DexIL, an imitation algorithm that trains dexterous robot skills from human hand mocap data.

What does DexCap capture?

Wrist and finger motion, tracked with SLAM and an electromagnetic field, plus 3D observations of the environment from a chest-worn RGB-D LiDAR camera.

Is DexCap open source?

The official GitHub repository provides collection, processing, dataset building and policy training code, and lists the MIT License. Dataset licence terms are Not stated in the sources read.

Is DexCap egocentric human video?

Not in the sense of video-only capture. It combines a chest-worn camera rack, hand-mounted tracking cameras and motion capture gloves, and its policy takes point clouds as input.

Does Firsthand use DexCap?

No. Firsthand is not part of it or affiliated with its authors, and makes no claim that DexCap uses Firsthand data or hardware or that any model was trained on Firsthand data.

Does Firsthand offer glove data?

The site states a separate pilot-ready tactile glove tier, time-synced to a head-mounted stereo camera by hardware timestamps and an IMU, with each glove calibrated every session.