What Is the Aria Everyday Activities Dataset?

AEA is introduced in Aria Everyday Activities Dataset (arXiv 2402.13349), with an official dataset page and documentation. The paper calls it an updated version of the Everyday Activities section of the Aria Pilot dataset. It is human recording data: the sources read do not describe robot actions.

AEA is third-party research from Meta Reality Labs Research. Firsthand is not affiliated with its authors and makes no claim that AEA uses Firsthand data or hardware, that Firsthand data is part of or compatible with it, or that any model was trained on Firsthand data. The site states custom egocentric collection to a written spec, fast and on demand: 3D hand pose at 21 joints per hand with a per-joint visibility flag, wrist trajectories and action segments. The only language field is one task instruction per episode.

Per the paper, each recording holds multimodal sensor data from the glasses plus outputs from the Project Aria Machine Perception Services (MPS): a 1 kHz open-loop 6DoF trajectory, a post-processed closed-loop trajectory sharing one reference frame across recordings in the same location, a semi-dense point cloud, and per-frame 3D eye gaze as a ray. Speech transcription is time aligned and comes from a separate proprietary service, not MPS. Per-frame camera and IMU calibration is included.

Per the paper, wearers followed five guidance scripts, each a story of a day, and improvised within them. Scenarios include cooking, daily cleaning, exercising, dining, reading, playing games and time with friends. Recordings used Project Aria Recording Profile 9; the table lists the sensors. GPS and WiFi were off. Multi-wearer recordings use SMPTE LTC timecode for sub-millisecond synchronization.

Paper Table 1 states 143 recordings and 7.3 hours with one or two wearers. The documentation page says over 7.5 hours, so read both.

The paper demonstrates two applications. One is 3D neural scene reconstruction with Gaussian splatting, using the closed-loop trajectories and point cloud. The other is prompted segmentation, where eye gaze or speech prompts EfficientSAM, with GroundingDINO handling the speech case. Reconstruction scores are not reproduced here. Robot learning is Not stated in the sources read.

The paper says recordings were captured in fully consented indoor environments, and that faces and licenses were manually annotated and blurred in the RGB and monochrome streams. Recordings that failed quality checks were excluded. The dataset page says access is by submitting an email and agreeing to the dataset licence agreement. Licence terms are Not stated in the sources read. Not legal advice; read the agreement.

All four are third-party research datasets, and Firsthand is not part of any of them. The paper says Ego4D and Epic-Kitchens were captured by head-mounted video devices and lack other sensor modalities needed to infer 3D information or wearer intention. The table sets only what each source states.

The sources read describe human daily activities and perception data. Hand pose annotations, contact labels and robot actions are Not stated in the sources read. The scripts are open-ended guidance, not a task spec for your robot.

AEA is a public research release, not a service. When you need specific tasks, environments or hand-pose fields, write them into a spec and confirm it on a Spec Pilot, a small fast first batch. The median from signed spec to first validated batch is 48 hours where coverage is deep and three to four weeks where it is thin. Billing is per validated hour; delivery is RLDS, LeRobot, WebDataset or HDF5.

What is the Aria Everyday Activities dataset?

An open egocentric multimodal dataset recorded with Project Aria glasses: 143 daily activity sequences by multiple wearers in five indoor locations, with 3D trajectories, a point cloud, per-frame 3D eye gaze and speech transcription.

How large is AEA?

Paper Table 1 states 143 recordings in five locations and 7.3 hours. The documentation page says over 7.5 hours.

What sensors and outputs does AEA include?

Per the paper: RGB, monochrome and eye-tracking cameras, IMUs, microphones, a magnetometer and a barometer, plus trajectories, a point cloud, 3D eye gaze and speech transcription.

Are faces anonymized in AEA?

The paper says faces and licenses were manually annotated and blurred in the RGB and monochrome streams, and recordings that failed quality checks were excluded.

Can AEA be used for robot training?

Not stated in the sources read. They describe human daily activities and perception research. Access requires agreeing to the dataset licence agreement, whose terms are Not stated in the sources read.

Does Firsthand use AEA, and can I request a custom dataset?

Firsthand is not affiliated with the AEA authors and makes no claim of use, training or compatibility. You can request a custom egocentric dataset: write a spec, confirm it on a Spec Pilot, pay per validated hour.