Glossary
What Is the First-Person Hand Action Benchmark (FPHA)?
Last reviewed: October 2026
Short answer
The First-Person Hand Action Benchmark (FPHA) is a published dataset of first-person RGB-D videos of daily hand actions with 3D hand pose annotations, introduced in arXiv 1704.02463. Its hand poses come from a motion-capture system with magnetic sensors, and its release terms are academic and non-commercial. It is third-party research, not Firsthand’s.
01
What is the First-Person Hand Action Benchmark (FPHA)?
FPHA is introduced in arXiv 1704.02463 by Guillermo Garcia-Hernando, Shanxin Yuan, Seungryul Baek and Tae-Kyun Kim, presented at CVPR 2018, with an official GitHub repository. The abstract says it studies the use of 3D hand poses to recognize first-person dynamic hand actions interacting with 3D objects, and calls it the first benchmark that enables that study. Read: the abstract and the GitHub README; not the full paper text. No separate official project page was read, so none is linked.
02
Where does Firsthand fit next to FPHA?
FPHA is third-party research. Firsthand is not affiliated with its authors and claims no use, training or compatibility with it. Firsthand captures real first-person human video, fast, on demand and to a written spec: 21 keypoints per hand with a per-joint visibility flag, wrist trajectories, contact events and action segments, plus one task instruction per episode.
03
How was FPHA captured, per the sources?
The abstract says the RGB-D video sequences hold more than 100K frames of 45 daily hand action categories, involving 26 different objects in several hand configurations. The README names an Intel RealSense SR300, with colour at 1920x1080 in JPEG and depth at 640x480, 16 bit, in PNG. Frame rates, hours of video, the number of subjects and how the camera was worn are Not stated in the sources read.
04
What hand pose, action and object annotations does FPHA provide, per the sources?
The abstract says the hand pose annotations come from the authors’ own mo-cap system, which infers the 3D location of each of the 21 joints of a hand model via 6 magnetic sensors and inverse kinematics. The README stores one line per frame, with world coordinates in millimetres for 21 joints running from the wrist through each finger. The abstract adds 6D object poses and 3D object models for a subset of the hand-object sequences; the README lists four such objects. Annotation accuracy is Not stated in the sources read.
05
What benchmarks or tasks does FPHA define, per the sources?
The abstract describes an evaluation of RGB-D and pose-based action recognition with 18 baselines or state-of-the-art approaches, several training and testing protocols, and an assessment of 3D hand pose estimation when objects severely occlude the hands. It concludes that hand pose is clearly beneficial as a cue for action recognition compared with other modalities. The README lists a 1:1 action recognition split and two hand pose protocols, cross subject and cross object. Scores are not reproduced here.
06
How does a lab hand-action benchmark differ from custom real-world egocentric task capture?
FPHA is a fixed dataset released after a form and agreement to terms: the README says it is for academic research only, free to researchers from educational or research institutes for non-commercial purposes, with no redistribution or commercial use without the authors’ express permission. Its hand poses come from magnetic sensors; Firsthand claims no marker-based or magnetic-sensor tracking. Firsthand is custom-collected to a written spec, confirmed on a Spec Pilot, and is not affiliated with FPHA’s authors.
07
What egocentric hand data can you get custom-collected on demand?
Firsthand captures real human first-person video, with no synthetic or simulated data. Each episode carries ten streams aligned under 2 ms, including a head camera, two wrist cameras, depth, an IMU, and 3D hand and body pose. The median from signed spec to first validated batch is 48 hours where coverage is deep and three to four weeks where thin. Formats: RLDS, LeRobot v2.1, WebDataset, HDF5.
08
FPHA capture setup, modalities and annotations, as stated in the official sources
| Item | What the official sources state |
|---|---|
| Publication | arXiv 1704.02463; CVPR 2018 (README) |
| Camera | Intel RealSense SR300 (README) |
| Colour stream | 1920x1080, 32 bit, JPEG (README) |
| Depth stream | 640x480, 16 bit, PNG (README) |
| Scale | More than 100K frames; 45 daily hand action categories; 26 different objects (abstract) |
| Hand pose | 21 joints from a mo-cap system with 6 magnetic sensors and inverse kinematics (abstract); world coordinates in mm (README) |
| Object pose | 6D object poses and 3D models for a subset of sequences (abstract); four objects with models in the README |
| Camera parameters | Intrinsics and extrinsics are listed in the README; values not reproduced here |
| Licence | Academic research only, non-commercial; no redistribution or commercial use without express permission (README) |
| Frame rate, hours, subject count | Not stated in the sources read |
| Firsthand relationship | None |
10
Tasks and evaluation named in the FPHA sources
| Item | What is stated |
|---|---|
| Action recognition | RGB-D and pose-based; a 1:1 split is listed in the README |
| Baselines | 18 baselines or state-of-the-art approaches (abstract) |
| Hand pose estimation | Cross-subject protocol, training on subjects 1, 3 and 4, and a cross-object protocol (README) |
| Occlusion study | How ready 3D hand pose estimation is when objects severely occlude hands in egocentric views (abstract) |
| Finding | Clear benefits of hand pose as a cue for action recognition compared with other modalities (abstract) |
| Metrics and scores | Not reproduced here |
11
What FPHA’s sources describe beside what the Firsthand site states
| Aspect | FPHA (sources read) | What the Firsthand site states |
|---|---|---|
| Origin | Fixed released dataset for academic, non-commercial research | Custom-collected to a written spec, on demand |
| Cameras | Intel RealSense SR300 RGB-D | Head-mounted egocentric video plus two wrist cameras |
| Depth | 640x480, 16 bit | Depth stream at 848 × 480 and 30 fps |
| Hand pose | 21 joints via magnetic sensors and inverse kinematics | 21 keypoints per hand with a per-joint visibility flag, in metric units in the head-camera frame, aligned to depth and RGB; no marker-based or magnetic-sensor tracking claimed |
| Object pose | 6D poses for a subset of sequences | Not claimed |
| Actions | 45 daily hand action categories | Action segments, contact events and one task instruction per episode |
| File layout | JPEG, PNG and text files per the README | RLDS, LeRobot v2.1, WebDataset, HDF5 |
| Compatibility | Not claimed | Not claimed |
12
Frequently asked questions
- What is the First-Person Hand Action Benchmark (FPHA)?
- A dataset of first-person RGB-D videos of daily hand actions with 3D hand pose annotations, introduced in arXiv 1704.02463.
- How were the FPHA hand poses obtained?
- Per the abstract, a mo-cap system infers the 3D location of 21 hand joints via 6 magnetic sensors and inverse kinematics.
- How big is FPHA?
- The abstract states more than 100K frames of 45 daily hand action categories involving 26 objects. Hour counts and subject totals are Not stated in the sources read.
- Is FPHA free for commercial use?
- The README says it is released for academic research only, free for non-commercial purposes to researchers from educational or research institutes, with no commercial use without express permission.
- Does Firsthand use magnetic sensors or markers for hand tracking?
- The site does not claim marker-based or magnetic-sensor tracking. It states 21 keypoints per hand with a per-joint visibility flag, aligned to depth and RGB.
- Does Firsthand use FPHA, and can I request a custom dataset?
- Firsthand is not affiliated with the FPHA authors and claims no use, training or compatibility. You can request a custom dataset, confirmed first on a Spec Pilot.
Related terms