What Is Ego-Exo4D?

Ego-Exo4D is described in the paper Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives (arXiv 2311.18259, first submitted in November 2023 and last revised in September 2024) and on its project page. The authors present it as a diverse, large-scale multimodal multiview video dataset and benchmark challenge, built to push first-person video understanding of skilled human activity. The video is accompanied by multichannel audio, eye gaze, 3D point clouds, camera poses, IMU and paired language descriptions. The project page states the dataset was released on 13 December 2023.

Ego-Exo4D is a third-party research dataset, and Firsthand is not part of it, endorsed by it or affiliated with its authors or their institutions. Firsthand captures egocentric video only: the site states it does not capture third-person or fixed external-camera footage, so it offers no exocentric views. What Firsthand states is custom egocentric collection: video from a head-mounted global-shutter camera plus two wrist cameras on one hardware-triggered clock, with depth, 3D hand pose and action segments, collected fast, on demand and to a written skill spec from paid, consented contributors, under a perpetual, buyer-owned commercial license. Firsthand makes no performance claim against Ego-Exo4D.

The abstract states that 740 participants from 13 cities performed skilled activities in 123 natural scene contexts, giving captures of 1 to 42 minutes each and 1,286 hours of video combined. The project page gives 5,035 takes across eight scenarios: cooking, music, soccer, health, basketball, dance, bike repair and rock climbing. The GitHub README for the V2 release states 1,286.30 video hours, of which 221.26 are ego-hours.

The project page lists three synchronized language datasets paired with the video: expert commentary that reveals nuanced skills, participant-provided narrate-and-act descriptions in a tutorial style, and one-sentence atomic action descriptions. The paper adds benchmark tasks and annotations for fine-grained activity understanding, proficiency estimation, cross-view translation and 3D hand and body pose.

Per the project page, the egocentric view comes from head-worn glasses with an 8 MP RGB camera and two SLAM cameras, calibrated and time-synchronized with 4 to 5 stationary exocentric cameras. The number and placement of the exocentric cameras is set per scenario to give maximal viewpoint coverage without obstructing the participant.

The other sensors it lists are an array of seven microphones, two IMUs (800 Hz and 1000 Hz), a barometer (50 fps) and a magnetometer (10 fps), plus machine-perception outputs: 6 DoF localization, eye gaze and point clouds.

The project page says Ego4D collected unscripted daily-life activities, while Ego-Exo4D focuses on skilled activities, with participants of diverse skill levels recorded performing the same scenario, and adds synchronized egocentric and multiple exocentric views. The GitHub README describes Ego4D as an egocentric video dataset with over 3,700 hours of annotated first-person video.

The project page states the dataset is publicly available: you sign the Ego-Exo4D licence forms, which it says take 2 days to approve, then download with a command-line downloader. The GitHub repository provides that downloader as the egoexo command and lists an MIT License for the repository itself.

The dataset licence terms are Not stated in the sources read beyond the signing requirement, so read the licence forms before planning any commercial use.

The project page states the videos were recorded under privacy and ethics policies with formal participant consent. Because the scenarios allow closed environments with no passersby, nearly all video is available without de-identification, and the page confirms the data contains identifying information of individuals.

Firsthand’s own approach, stated elsewhere on this site, is a signed consent chain with faces and identifying details outside the task anonymized.

Write the target skills, environments and signals into a skill spec and confirm them on a Spec Pilot, a small fast first batch. This is on-demand custom collection, not a catalog download.

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is Ego-Exo4D?

A multimodal, multiview video dataset and benchmark challenge of skilled human activity, recorded as simultaneous egocentric and exocentric video.

How large is Ego-Exo4D?

The official sources state 740 participants, 13 cities, 123 scene contexts, 5,035 takes and 1,286 hours of video combined.

Is Ego-Exo4D egocentric video only?

No. It pairs egocentric video with several stationary exocentric cameras, plus audio, IMU and eye gaze.

How do you get Ego-Exo4D?

Sign the licence forms, which the project page says take about 2 days to approve, then download with the command-line downloader. The licence terms themselves are Not stated in the sources read.

Does Firsthand capture exocentric or third-person video?

No. The site states Firsthand captures egocentric video only and does not capture third-person or fixed external-camera footage.

Is Firsthand data part of Ego-Exo4D?

No. Firsthand is not part of it, endorsed by it or affiliated with its authors, and makes no performance claim against it.

How fast can custom egocentric data be collected?

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks.