Firsthand vs Ego4D

Short answer

Ego4D is a large, free research corpus of unstructured daily activity whose license permits commercial use, so the honest gap is fitness, not legality: it lacks manipulation-grade 3D hand pose and frame-accurate action labels. Firsthand captures to a named skill spec with synchronized depth and pose.

Side by side

Firsthand vs Ego4D, dimension by dimension.

Dimension	Firsthand	Ego4D
Hours	Custom to your spec; 10,585 validated hours across the current catalog	~3,670 hours of daily activity [VERIFY]
Capture setup	Head + two wrist cameras, active-stereo depth, 3× IMU, hardware-triggered on one PTP clock	Many head-mounted rigs across a multi-site consortium [VERIFY]
Streams	Ten synchronized streams incl. depth, IMU, 3D hand & body pose, segmentation, action segments	RGB; subsets add audio, IMU, 3D scans, gaze by scenario [VERIFY]
Sync	Hardware-triggered; median 1.12 ms, 2.0 ms ceiling, drift written to metadata	Not hardware-synchronized to a sub-2 ms ceiling [VERIFY]
Hand pose	3D, handJoints[2][21][3], metric, in the head-camera frame	No manipulation-grade 3D hand pose
Depth	Metric depth, 848×480 at 30 Hz, with per-pixel confidence	Scenario-dependent, not universal metric depth [VERIFY]
Action labels	Frame-accurate action segments (verb, noun, ns timestamps)	Dense narrations and task benchmarks; not frame-accurate manipulation segments throughout [VERIFY]
Failure-case coverage	Transit, occlusion and low-light shipped — not trimmed — with a published reject log	Broad in-the-wild activity, not curated to a manipulation skill
License	Perpetual, irrevocable, buyer-owned commercial license	Ego4D Dataset License Agreement [VERIFY]
Commercial use allowed	Yes	Permitted under the Ego4D license terms [VERIFY]
Consent documentation	Written release per participant; consent artefacts + data card per batch	Governed by the consortium consent process [VERIFY]
Delivery formats	RLDS, LeRobot, WebDataset, HDF5, zarr, Rerun .rrd	Ego4D distribution format and tooling [VERIFY]
Cost	Per validated hour, quoted per engagement; free sample pack, no form	Free under license terms

Third-party figures marked [VERIFY] are confirmed against the sources below before publishing. Firsthand figures are measured on-site.

When Ego4D is the right choice

You need broad, cheap pretraining scale across many everyday activities.
Academic research where the Ego4D license and benchmarks fit your use.
You want a public baseline everyone in the field already knows.

When Firsthand is the right choice

You are training a manipulation or whole-body policy and need 3D hand pose and metric depth.
You need frame-accurate action segments against a named skill.
You need a clean, buyer-owned commercial license and documented consent per participant.

FAQ

Common questions.

Is Ego4D allowed for commercial use?

The Ego4D Dataset License Agreement does permit commercial use subject to its terms [VERIFY]. The practical constraint is fitness, not legality — it is unstructured daily activity without manipulation-grade hand pose.

Does Ego4D have 3D hand pose?

Not manipulation-grade 3D hand pose across the corpus [VERIFY]. Some subsets carry richer signals, but it was not built as spec-fit manipulation data.

Can I combine Ego4D and Firsthand?

Yes, and teams do — Ego4D for broad pretraining scale, Firsthand for spec-fit fine-tuning with pose, depth and action segments. Delivery in RLDS and LeRobot makes mixing straightforward.

Last reviewed: 2026-09-23

Sources

Ego4D official site
Ego4D license terms [VERIFY]

Judge it on your own skill spec.

Get the free 40-episode sample pack, or send us your target skills and hours for a scoped quote.

Get the free sample pack →Request a quote