Glossary
What Is Human Demonstration Data?
Last reviewed: September 2026
Short answer
Human demonstration data is a recording of a person performing a task directly — not a robot driven by an operator — captured to train a policy by imitation learning. In egocentric form it is recorded from a first-person, worn viewpoint rather than a third-person camera, alongside signals like depth, hand pose and action labels.
01
Why it matters
A policy trained by imitation learning is only as good as the demonstrations it copies. Human demonstration data scales coverage across tasks, sites and people faster than a robot-and-operator fleet can, at the cost of an embodiment gap between a human hand and a robot end-effector that retargeting or pairing must close.
02
How Firsthand handles it
Firsthand captures human demonstrations from the first-person, egocentric viewpoint with ten synchronized streams — stereo video, depth, IMU, 3D hand pose and frame-accurate action segments — hardware-triggered on one clock, delivered under a buyer-owned commercial license with a documented consent chain.
03
What a Firsthand human demonstration episode contains
| Signal | What it captures | Where it lives in the episode |
|---|---|---|
| Stereo video | The demonstrator’s first-person view of the task and environment | Head-mounted global-shutter 4K stream |
| Depth | Metric distance to the scene, derived from the stereo rig | Per-frame depth stream, hardware-synced to video |
| IMU | Motion and orientation of the head and wrists | Per-device inertial stream on the same clock |
| 3D hand pose | Finger and wrist geometry during the task | Per-frame hand-pose stream |
| Action segments | Frame-accurate labels for what the demonstrator is doing | action_segment field per step |
| Provenance & consent | Written release and per-episode consent artefact | Data card and provenance metadata for the batch |
05
Frequently asked questions
- Is human demonstration data the same as teleoperation data?
- No. Human demonstration data records a person performing a task directly; teleoperation records a human operator driving a robot. Human capture scales faster and more broadly, while teleoperation matches the target robot’s embodiment exactly. See teleoperation vs egocentric data for the full comparison.
- What is the "embodiment gap" in human demonstration data?
- A human hand and a robot end-effector have different kinematics, degrees of freedom, and sensor placement. Data captured from a human body doesn’t map one-to-one onto robot joint commands, so a retargeting or alignment step is typically needed to close that gap.
- What signals does human demonstration data usually include?
- At minimum, video of the task and some record of the actions performed. Firsthand’s captures add depth, IMU, 3D hand pose and frame-accurate action segments, hardware-synchronized onto one clock, so a policy gets richer supervision than video alone.
- Does human demonstration data need consent documentation?
- Yes — it records a real person performing a task, so a written release and a per-episode consent artefact are part of the dataset’s provenance. See dataset provenance explained and consent requirements for AI training data for the full picture.
- Can human demonstration data be used to train a commercial model?
- It depends on the license the data was collected under. Firsthand delivers human demonstration data under a buyer-owned commercial license with a documented consent chain; publicly released datasets vary and should be checked individually. See research-only vs commercial dataset licenses.