Egocentric-10K vs EgoVerse for Robot Pretraining

Egocentric-10K is a published dataset hosted on Hugging Face by builddotai at huggingface.co/datasets/builddotai/Egocentric-10K. Its dataset card states roughly 10,000 total hours of egocentric video, 1.08 billion frames and 192,900 video clips with a median clip length of 180 seconds, collected exclusively inside real factories across 85 factory sites (factory_001 through factory_085).

Each clip is captured with a single device the card calls Build AI Gen 1 — a monocular, head-mounted camera recording 1080p (1920×1080) H.265/MP4 video at 30 fps with a 128° horizontal by 67° vertical field of view and no audio track. The card reports a mean of 4.68 hours of footage per worker and a total storage size of 16.4 TB, structured as WebDataset-format TAR shards organized by factory and worker, each paired with a per-worker intrinsics.json file carrying calibrated fisheye camera parameters (the OpenCV fisheye / Kannala-Brandt model with four distortion coefficients).

The card states the dataset is licensed under Apache 2.0. See /guides/what-is-egocentric-10k for the full single-dataset explainer sourced strictly from the same card.

EgoVerse is described in the paper "EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World" at arXiv 2604.07607. The paper reports roughly 1,362 hours of egocentric human demonstrations across approximately 80,000 episodes, spanning 1,965 tasks, 240 scenes and 2,087 unique demonstrators, contributed by a consortium of academic labs and industry partners.

The dataset is split into two components with different purposes. EgoVerse-A is collected under a standardized protocol mirrored across participating academic labs using Project Aria Gen 1 glasses, and is designed for controlled, reproducible study across six shared "flagship" manipulation tasks. EgoVerse-I is sourced from industry partners using custom wearable rigs and a phone-based capture pipeline, contributing the larger share of total hours and nearly 2,000 open-ended tasks, with denser, language-based annotations.

Per-frame annotations include 3D hand pose using 21 keypoints per hand, paired with a calibrated 6-DoF head pose from visual-inertial SLAM. EgoVerse-I episodes add fine-grained (1–2 second) language descriptions, active-hand indicators and static-versus-mobile manipulation flags. The paper does not state a license for the dataset anywhere in the text this site cites. See /guides/what-is-egoverse for the full single-dataset explainer.

On raw hours, Egocentric-10K is the larger corpus at roughly 10,000 hours against EgoVerse’s roughly 1,362 hours — about seven times more footage. On task and scene diversity, EgoVerse is the broader corpus: its paper documents 1,965 tasks and 240 scenes and 2,087 demonstrators, against a single domain (factory work) for Egocentric-10K, whose card does not break coverage out by task.

Capture hardware differs by design intent. Egocentric-10K standardizes on one device (Build AI Gen 1) for every worker, which keeps resolution, frame rate and field of view identical across the whole corpus. EgoVerse deliberately mixes three capture systems — Project Aria glasses, custom industry rigs, and a phone-based pipeline — which broadens the range of environments and demonstrators captured but means the paper does not state one dataset-wide resolution or frame-rate spec.

Annotation depth points the other way. Egocentric-10K’s card documents calibrated camera intrinsics per worker but no action, hand-pose or language labels. EgoVerse’s paper documents 3D hand pose and head pose for every episode, plus language descriptions for its EgoVerse-I component — annotation types a pretraining pipeline that trains on hand or language-conditioned signals would need to add itself for Egocentric-10K.

The two tables below lay out the attribute-by-attribute comparison, followed by how each gap maps to what a custom collection covers for a startup’s specific target task.

Egocentric-10K’s Hugging Face dataset card states an Apache 2.0 license. Apache 2.0 is a widely used permissive license that allows use, modification and redistribution, including for commercial purposes, subject to the license’s own terms — see the official license text at apache.org/licenses/LICENSE-2.0 for exactly what it permits and requires.

EgoVerse’s arXiv paper does not state a license for the dataset anywhere in the text this site cites. That is a gap, not a defect in the paper — many research datasets are published without a license statement in the paper itself, and the terms (if any) would need to be confirmed directly with the dataset’s maintainers before any commercial use.

For an early-stage robotics team, an unconfirmed license on a human-demonstration dataset is a real blocker for a funded product, not a detail to defer. A custom collection closes that gap directly: Firsthand’s default is a buyer-owned, perpetual, non-exclusive commercial license, confirmed in writing before capture begins rather than inferred from a paper that does not address it. See /guides/buyer-owned-training-data for why that distinction matters for a product roadmap.

Egocentric-10K’s gap is domain breadth and annotation depth. Its card states the footage is collected exclusively inside real factories, so a team pretraining for a household, retail, office, warehouse-logistics-adjacent-but-different, or outdoor task has no coverage of that domain in this corpus. The card also documents camera intrinsics only — no action labels, hand-pose tracks or language annotations — so any such signal would need to be produced by the team itself from raw video, rather than consumed directly.

EgoVerse’s gap is licensing certainty and absolute scale. Its paper states no license, which means a commercial program cannot rely on the paper alone to clear legal review. At roughly 1,362 hours, EgoVerse also sits toward the smaller end of published egocentric pretraining corpora this site tracks on /compare/egocentric-pretraining-datasets, so a team planning a large pretraining run may need to combine it with other sources regardless of license status.

Both corpora share one structural limitation for robot learning specifically: neither is robot execution data. EgoVerse’s own paper reports that co-training with human data improves robot performance only when that human data aligns with the target robot’s task semantics — a finding that applies equally to Egocentric-10K’s factory footage. Public human-video corpora are a starting point for a visuomotor prior, not a substitute for data collected against a specific robot’s own task distribution.

Yes, and that is how both corpora are typically used in practice. A team can pretrain broadly on Egocentric-10K’s factory-floor manipulation footage and EgoVerse’s broader task and scene diversity, then commission a custom, commercially licensed batch scoped to the exact tasks, objects and environments its own robot still needs — the same pattern this site documents for other public corpora on /compare/public-vs-custom-pretraining-datasets.

A custom collection for a startup typically starts small: a written skill spec covering the scenes, tasks, objects and people a public corpus is missing, proven on a Spec Pilot before scaling volume, as described on /custom-collection/egocentric-data-collection-pilot. That lets an early-stage team confirm coverage and format fit on a small first batch before committing to a larger production order.

Firsthand delivers custom batches in RLDS, LeRobot, WebDataset, HDF5, zarr and Rerun .rrd, so a custom collection mixes cleanly alongside data already referenced from Egocentric-10K or EgoVerse at any pipeline stage — pretraining, mid-training or fine-tuning. See /guides/robot-pretraining-data for how those stages divide the work, and /custom-collection/custom-egocentric-dataset for what a scoped custom order includes.

Use Egocentric-10K when the priority is a large, permissively licensed corpus and the target task is reasonably close to factory-floor manipulation — picking, placing and handling objects in a structured industrial setting. Its stated Apache 2.0 license is the clearest commercial-use license of the two, confirmed directly on the dataset card rather than left unstated.

Use EgoVerse as a reference point for task, scene and demonstrator diversity, and for its documented hand-pose and language annotations, while treating its lack of a stated license as a open question to resolve before any commercial training run — not an assumption to make on the paper’s behalf.

For most early-stage robotics teams, the realistic answer is both as a broad pretraining base, plus a custom collection scoped to whatever target tasks neither open corpus covers and under a license confirmed in writing from the start. Request a custom dataset at /custom-collection/pretraining-data-collection to scope that gap against your own skill spec.