What Is GigaHands?

GigaHands is introduced in GigaHands: A Massive Annotated Dataset of Bimanual Hand Activities (arXiv 2412.04244), a CVPR 2025 paper that arXiv and the README mark as a Highlight. It has an official project page and an official GitHub repository. The authors argue that large models of bimanual activity cannot yet be built because existing datasets lack scale, coverage of diverse hand activities and detailed annotations. Their answer is a markerless capture setup and an acquisition protocol that allow fully automatic 3D hand and object estimation while limiting the effort spent on text annotation. The abstract names three applications: text-driven action synthesis, hand motion captioning and dynamic radiance field reconstruction. The project page also shows manipulation sequences retargeted to robot grippers with a separate method, ManipTrans. This page draws only on the paper, the project page and the README. GigaHands is human hand data recorded in a capture rig, so robot actions are Not stated in the sources read, and the paper lists robotic manipulation as future work.

Per the paper, the data comes from a custom multi-camera tabletop capture system of 51 RGB cameras arranged in a cubic capture volume, with a 3×3 grid of cameras on each face, even LED lighting and a transparent glass surface that supports objects. Each camera records at 30 fps and 1280×720. Cameras are software-synchronized with under 3 ms of temporal misalignment, and intrinsics and extrinsics come from COLMAP aided by fiducial markers. Capture is markerless. The authors call it a studio setting and say it confines recording to a limited space. Subjects follow scripted instructions, played as audio, and keep both hands in the filming area. The scripts, which an LLM helped group and order, cover 5 scenarios, 25 scenes, 191 activities and 1,370 instructions. Annotators then split the 13k recorded sequences into 14k clips and annotate actions not in the script. The viewpoint is multi-view from the 51 cameras; the sources read do not describe GigaHands as egocentric. Face anonymization is Not stated in the sources read.

The project page and paper list the same set for every sequence: detailed activity text descriptions, 3D hand shape and pose, 3D object shape, pose and appearance, hand and object segmentation masks, 2D and 3D hand keypoints, and camera pose. Hands are given as 3D keypoints and MANO hand meshes, which the paper counts as 3.7 million bimanual 3D hand poses. Of the 417 objects, 310 have multi-view scanned meshes and 31 have single-view generated meshes, and masks cover non-rigid objects. The paper says fully automatic tracking of articulated and non-rigid objects remains challenging.

Text comes from an instruct-to-annotate strategy. Each filmed sequence is paired with its instruction, annotators correct and extend it, and an LLM rephrases each description five times, expanding 14k clips into 84k motion-text pairs with 1,467 unique verbs, so each clip carries six descriptions. The README says the rewritten_annotation field was used for model training. The paper trains text-to-motion and captioning models on the data and reports scores against other datasets; those scores are not reproduced here. Per the README, released parts include hand poses, multiview RGB videos, object poses for 3.3k motion sequences, object meshes, text annotations and text-to-motion code, hosted on Globus. The project page still shows “To be released soon” under Dataset Annotations, so the README is the better guide to release status.

The paper opens by calling the understanding of bimanual human hand activity a critical problem in AI and robotics, and says enabling machines to understand and replicate human hands is a grand challenge. It contrasts the data behind recent large models, which include millions of robot trajectories, with the lack of comparable data for natural hand activity. It notes that occlusion makes reliable 3D motion for both hands and objects hard to obtain in the wild, and presents its dense markerless multi-camera setup as a balance of accuracy, diversity and practicality.

Robot learning is not the paper’s target: its demonstrated applications are motion synthesis, captioning and radiance fields. Training robot policies on GigaHands is Not stated in the sources read, and any robot use is the reader’s own judgement. What bimanual hand data does supply is two-handed manipulation signal, the kind of human demonstration signal embodied-AI teams look for.

GigaHands pairs a 51-camera markerless tabletop rig with dense text. The other three are described by their site entries: HOT3D is an egocentric multi-view dataset with marker-based motion-capture ground truth, ARCTIC is a motion-capture dataset of two hands with articulated objects, 8 allocentric views and 1 moving egocentric view, and DexYCB is a hand-grasping benchmark whose camera count and viewpoint are Not stated in the sources that entry read.

The first comparison table uses only the GigaHands sources and those existing site entries, so a cell where an entry is silent says so. The GigaHands paper reports its own comparison with other datasets; this page does not reproduce it and ranks none of the four. Firsthand is not part of any of them.

GigaHands is a public research release, not a service. Firsthand is not affiliated with its authors and makes no claim of use, training or compatibility with it. If you need your own tasks and environments, the site states custom egocentric collection to a written spec, fast and on demand: first-person human video with 3D hand pose at 21 joints per hand and a per-joint visibility flag, wrist trajectories and action segments, captured with a head-mounted camera plus two wrist cameras on one hardware-triggered clock. Both hands stay in frame with per-hand 3D pose, including long continuous clips of fine bimanual work. The only language field is one task instruction per episode.

Confirm a spec on a Spec Pilot, a small fast first batch. The median from signed spec to first validated batch is 48 hours where coverage is deep and three to four weeks where it is thin. Billing is per validated hour; delivery is RLDS, LeRobot, WebDataset or HDF5. If contact signals matter, ask for the separate pilot-ready tactile glove tier. This page makes no claim that Firsthand provides a multi-camera studio rig, MANO hand meshes, object meshes or 3D object tracking, or text motion descriptions.

What is GigaHands?

A published dataset of bimanual hand activities from Brown University and ETH Zurich researchers, with 3D hand and object annotations and paired text, presented at CVPR 2025.

How large is GigaHands?

The abstract states 34 hours from 56 subjects and 417 objects, with 14k motion clips from 183 million frames and 84k text annotations. The paper body gives 2,034 minutes.

How many cameras does GigaHands use, and is it egocentric?

The paper describes 51 RGB cameras at 30 fps and 1280×720 in a cubic tabletop capture volume. The sources read do not describe it as egocentric.

What does GigaHands annotate?

Text descriptions, 3D hand shape and pose, 3D object shape, pose and appearance, segmentation masks, 2D and 3D hand keypoints and camera pose, with MANO hand meshes.

Can GigaHands be used commercially?

The README says it is released under CC BY-NC 4.0, a non-commercial licence. Read the licence text; this is not legal advice. Other terms are Not stated in the sources read.

Does Firsthand use GigaHands, and can I request a custom dataset?

Firsthand is not affiliated with the GigaHands authors and makes no claim of use, training or compatibility. You can request a custom egocentric dataset: write a spec, confirm it on a Spec Pilot, pay per validated hour.