What Is the Nymeria Dataset?

Nymeria is introduced in Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild (arXiv 2406.09905). The abstract calls it a large-scale, richly annotated human motion dataset, with all devices synchronized and localized in one metric 3D world. It is human motion data: the sources read do not describe it as robot data.

Nymeria is third-party research. Firsthand is not affiliated with its authors and makes no claim that Nymeria uses Firsthand data or hardware, or that any model was trained on Firsthand data. The site states custom egocentric collection to a written spec, fast and on demand: 3D hand pose at 21 joints per hand with a per-joint visibility flag, wrist trajectories and action segments. The only language field is one task instruction per episode.

Per the project page, participants wear a Project Aria headset recording RGB, grayscale and eye-tracking video, IMUs, a magnetometer, a barometer and audio. miniAria wristbands capture egocentric data from the wrist, an XSens MVN Link suit with 17 inertial trackers records full-body kinematics, and an observer with a second Aria headset gives a third-person view. The abstract states 300 hours, 264 participants and 50 locations.

The abstract names full-body ground-truth motion, multimodal egocentric streams and the observer view. Language is hierarchical: motion narration, simplified atomic action and high-level activity summarization, written by annotators watching synchronized playback. Sentence totals differ between the abstract and the project page, so read both. The project page also announces NymeriaPlus, with added object boxes and modalities. Evaluation scores are not reproduced here.

The project page and README state CC BY-NC 4.0, and the README says data and code may not be used for commercial purposes. The project page says participants and home owners gave consent and that EgoBlur blurred faces and license plates in all videos. Not legal advice; read the licence files.

All three are third-party research datasets, and Firsthand is not part of any of them. Nymeria centres on whole-body daily motion, Ego-Exo4D on skilled activity and HOT3D on hand and object tracking. The table sets only what each source states.

The sources read describe human body motion and 20 common daily scenarios, not robot actions or a manipulation task spec. Contact labels are Not stated in the sources read, and whether a robot model can learn from Nymeria is also Not stated. The stated licence bars commercial use, so programs needing specific tasks must specify them separately.

Nymeria is a public research release, not a service. Write the tasks, environments and people you need into a spec and confirm it on a Spec Pilot, a small fast first batch. The median from signed spec to first validated batch is 48 hours where coverage is deep and three to four weeks where it is thin. Billing is per validated hour; delivery is RLDS, LeRobot, WebDataset or HDF5.

What is the Nymeria dataset?

A research dataset of multimodal egocentric daily motion in the wild, with full-body ground-truth motion, Project Aria data, a third-person observer view and hierarchical language descriptions.

How large is Nymeria?

The paper abstract states 300 hours from 264 participants across 50 locations. Some totals differ between the paper, project page and README.

What sensors were used?

Per the project page: Project Aria headsets, miniAria wristbands, an XSens MVN Link suit and an observer headset.

Can Nymeria be used commercially?

The sources state CC BY-NC 4.0, and the README says data and code may not be used for commercial purposes. This is not legal advice.

Does Nymeria include robot actions or contact labels?

The sources read describe human motion, not robot actions. Contact labels are Not stated in the sources read.

Does Firsthand use Nymeria, and can I request a custom dataset?

Firsthand is not affiliated with the Nymeria authors and makes no claim of use, training or compatibility. You can request a custom egocentric dataset: write a spec, confirm it on a Spec Pilot, pay per validated hour.