Guides

Guides for collecting and training on egocentric data

Standalone, practical write-ups of the things teams ask before a capture program: how to spec a skill, how sync is measured, how much data a policy actually needs, and how to license it for commercial use.

What Is NVIDIA EgoScale?A plain-language answer-engine explainer of what EgoScale is, what it found, and what its limitations mean for a pretraining plan.
Why Failure Cases Matter in Robot LearningWhy deliberately capturing transit, occlusion, low light and recovery attempts is what makes training data teach robustness instead of a best-case demo.
How to Capture Failure Cases in Robot Training DataA practical guide to planning, capturing, labeling and QA-checking failure and recovery behavior in egocentric robot training data, instead of leaving it to chance.
Action Segmentation for Robot LearningHow egocentric video is broken into labeled sub-action segments, and how to spec verb-noun boundaries a policy can actually learn from.
Skill-Level vs Frame-Level Annotations for Robot LearningHow segment-level action labels differ from per-frame labels like hand pose and masks, and which training stage needs which.
3D Hand Pose Estimation for RoboticsHow 3D hand pose is recovered from egocentric video, why manipulation policies need it, and how to spec it as validated per-hand joint data.
Hand Pose and Wrist Action Labels for Robot PretrainingHow wrist motion and articulated hand pose become the action label in human-video pretraining, and which fields Firsthand ships.
What Makes Robotics Training Data High Quality?A hub covering the concrete signals — sync error, validated hours, failure coverage, consent — that separate high-quality robotics training data from raw footage.
Multi-Layer QA for Video DatasetsHow multi-layer QA checks a video dataset’s sync, coverage, and consent at capture time, not after delivery.
What Are Rejection Criteria for Robotics Training Data?A criterion-by-criterion look at what fails a clip or episode before delivery and what happens to it next.
Robot Pretraining Data Quality ChecklistA quality checklist scoped specifically to pretraining-size corpora — tens of thousands of hours across many scenes, tasks, objects and people — rather than a single task dataset.
Robotics Dataset Quality ChecklistA practical, pass/fail checklist for evaluating any robotics training dataset against the measured signals that separate usable data from noisy footage.
How to Collect Egocentric Video DataA methodology walkthrough of how egocentric video data collection actually works, from spec to delivery — for teams evaluating how to source or commission it.
How to Collect Robot Pretraining DataA step-by-step walkthrough of how robot pretraining data is actually collected, from writing the spec through validated delivery, for teams planning a pretraining collection.
Egocentric Video Data for Embodied AIA hub covering what egocentric video is, why embodied AI needs it, and how to source it — public, custom, or on demand.
Can EgoDex Be Used Commercially?EgoDex is released under CC BY-NC-ND, which prohibits both commercial use and derivative redistribution.
Can Ego4D Be Used Commercially?Ego4D permits commercial use under its own dataset agreement — but check the terms and the fit, not just the license name.
How to write a skill spec for egocentric data collectionName the task, geometry, condition budget, reject rule and one acceptance test.
How to Capture Enterprise Workflows for AI TrainingA step-by-step methodology for turning enterprise SOPs into skill specs, piloting capture, and shipping QA’d workflow data for AI training.
How to measure cross-stream sync error in egocentric captureHardware-trigger every stream onto one PTP clock, then measure the residual offset.
Why Camera Synchronization Matters in Robotics DataUnsynchronized streams silently corrupt supervision at exactly the moments that matter most: fast contact and grasp.
Camera calibration procedure for a multi-camera egocentric rigOpening and closing solves, sub-0.28 px reprojection, drift measured per episode.
How to Calibrate Head and Wrist Cameras TogetherIntrinsics, extrinsics, PTP sync, and the reprojection gate for a head + wrist rig.
How to Set Up a Stereo Camera Rig for Robotics Data CollectionCameras, mounting, calibration, sync, and depth for an egocentric stereo rig.
Stereo Egocentric Data for Construction RoboticsA hub for stereo egocentric construction data: crews, machinery, finishing, marking and taping, inspection, MCAP delivery and recurring supply.
Evaluation methodology for egocentric manipulation datasetsHeld-out environments, matched hour budgets, and an open evaluation protocol.
Python quickstart for loading egocentric training dataInstall, list access, pull an episode, and stream aligned frames into training.
Egocentric Video Data for World ModelsDense, causal, first-person video with the action and pose stream that lets a model learn cause and effect.
Training Data for Vision-Language-Action (VLA) ModelsEgocentric video, task instruction, and a synchronized action stream in one episode — the three inputs a VLA action head needs.
How Frontier AI Labs Source Robotics Training DataA comparison of the four ways frontier AI labs source real-world robotics training data, and when each fits.
How to Evaluate a Robotics Data VendorA step-by-step due-diligence process and scorecard for evaluating a robotics data vendor.
How to Choose an Egocentric Data ProviderA neutral checklist for evaluating egocentric data vendors against your own spec.
How to Buy Egocentric Video DataA buyer-side walkthrough of sourcing routes, specs, pilots, QA and licensing for an egocentric video purchase.
Stereo vs Monocular Video for Robot LearningCalibrated dual-camera depth vs single-camera capture — and how to choose for your policy.
Depth Data for Embodied AIHow a calibrated stereo rig turns disparity into metric depth, and how that depth stream ships alongside RGB, IMU, and hand pose.
Using IMU Data in Robot Learning DatasetsHow Firsthand captures 200 Hz inertial data on the same hardware clock as RGB, depth, and hand pose, and what the delivered IMU stream contains.
Robotics Training Data ValidationThe validation process itself — the checks a raw hour of footage has to pass before it counts as a validated hour.
Validated Hours vs Raw Capture HoursA validated hour passed QA against your spec; a raw hour is unfiltered recording that may not.
How much egocentric data do you need to train a robot policy?Start around 50–100 validated hours per skill, then scale where evaluation is thin.
How Much Pretraining Data Does a Robot Foundation Model Need?Published corpora span 1,362 to 270,000 hours; diversity and validated hours matter more than raw count.
How Much Training Data Does a Humanoid Robot Need?No single number: public humanoid and robot pretraining corpora span three orders of magnitude, and diversity matters as much as hours.
How to Build a Robot Pretraining Data MixtureMixing egocentric human video with synthetic and real robot data, using the GR00T N1 pyramid and GEN-0's diversity findings.
What Is the GR00T N1 Data Pyramid?How NVIDIA GR00T N1 structures pretraining data as a pyramid, and what each layer contributes.
Egocentric data for humanoid robots: what to collectFirst-person, bimanual demonstration with 3D hand pose maps cleanly to humanoids.
Humanoid Robot Data Collection Methods ComparedA comparison of the main ways teams gather humanoid robot training data, and where each fits.
Perpetual vs Term Licenses for AI Training DataA perpetual license never expires; a term license grants rights only for a fixed window, after which continued use of the data — and sometimes the model trained on it — can become unauthorized.
How to license training data for commercial modelsCheck consent documentation and for a perpetual, buyer-owned commercial license.
How to License Human Demonstration DataHuman demonstration data needs a commercial-use license with documented per-participant consent, not just a large, well-labeled corpus. What to check before training on it.
AI Training Data Rights ChecklistBefore training a commercial model on any dataset, confirm commercial-use rights, a documented consent chain, verified provenance, anonymization, and perpetual buyer-owned ownership terms.
Research-Only vs Commercial Dataset LicensesResearch-only licenses block commercial training outright; a commercial or buyer-owned license is what lets a team actually ship a trained model.
Buyer-Owned Training Data ExplainedBuyer-owned means a perpetual, irrevocable commercial license backed by a documented consent chain, not a revocable subscription to someone else’s dataset.
How AI Labs Keep Custom Training Data ExclusiveExclusivity comes from a buyer-owned license, a documented consent chain, and a contract that says so — not from secrecy alone.
Consent Requirements for AI Training DataA written release before capture, a recorded legal basis, and a per-episode artefact trail are what make training data consent checkable rather than assumed.
How to blur faces in video automaticallyDetect each face as a box, track it across frames, blur it into the pixels, then review and certify.
Face detection and bounding boxes, explainedWhat a detector returns, how the confidence threshold works, and why it is only step one.
How to Anonymize First-Person (Egocentric) VideoEgocentric video needs the same detect-track-redact-review pass as any footage, but it must leave the wearer’s hands, gaze and the geometry of the scene untouched — because that is the training signal the data was collected for.
Video anonymization and privacy law: GDPR, CCPA and de-identificationWhat counts as personal data in footage, and why anonymization must be irreversible to fall out of scope.
Personally Identifiable Information in Robotics DataFaces, plates, screens and speech are the PII types that show up in egocentric capture, and each has a specific handling step.
Computer Vision Training Data for RoboticsWhy robotics computer vision needs synchronized, calibrated multi-stream video, not a labeled image set.
How Embodied AI Models Are TrainedThe training data pipeline behind embodied AI models, from egocentric capture to imitation learning and VLA policies.
Robot Learning From Human VideoWhat it takes for human video to actually teach a robot a task, and why most internet footage falls short.
Is Egocentric Human Video Good Pretraining Data for Robots?A direct, evidence-based answer to whether egocentric human video is good pretraining data for robot foundation models, and where it falls short.
How to Plan Diversity in a Robot Pretraining DatasetA practical planning guide for writing diversity coverage targets into a pretraining data collection spec, and checking coverage during a pilot before scaling.
How Does Pretraining Data Affect Robot Generalization?A mechanism explainer of how pretraining data composition — scale, diversity, label quality, failure coverage — drives a robot policy’s out-of-distribution generalization.
Why Dataset Diversity Matters in Robot LearningWhy a narrow, repetitive dataset teaches a policy that only works in the conditions it was filmed in, and how a to-spec program mix fixes that.
Training Data for Household RobotsWhy household robots need long-horizon, low-precision, high-navigation episodes across a whole home, and how that differs from tabletop manipulation data.
How to Convert RLDS to LeRobot FormatHow RLDS step and episode keys map onto LeRobot v2.1 features, and what to verify after conversion.
Dataset Provenance ExplainedWhat dataset provenance covers, what a per-episode record needs to carry, and how it relates to consent and licensing.
Long-Horizon Task Data for Robot PretrainingWhat long-horizon, multi-step demonstrations are, why pretraining needs them, and how action segments, episodes and skill specs make them trainable.
What Is Robot Pretraining Data?The pillar guide to pretraining data for robot foundation models: what it is, how EgoScale and GR00T N1 use it, and how to get a custom pretraining dataset.
Do Robot Foundation Models Follow Data Scaling Laws?What EgoScale’s log-linear scaling law and GEN-0’s scaling findings say about robot foundation model pretraining, and what scaling laws do and don’t tell you.
How to Collect Aligned Human-Robot Mid-Training DataA step-by-step methodology for collecting the small, embodiment-matched human-and-robot dataset used in the mid-training stage, between broad pretraining and task-specific fine-tuning.
How Does Human-to-Robot Transfer Work in Robot Learning?A mechanism-level explainer of how skills learned from human egocentric video transfer into a robot policy, from action representation through the three-stage training recipe.
Pretraining vs Mid-Training vs Post-Training Data for RobotsA stage-by-stage breakdown of pretraining, mid-training and post-training data for robot foundation models, and how to plan a custom dataset for each.
How to License Pretraining Data for Commercial Robot ModelsMost public egocentric pretraining datasets are research-only. A commercial robot model needs data whose license explicitly grants commercial rights and whose consent is documented per participant.
How to Choose a Robot Pretraining Data ProviderA buyer’s checklist and questions to ask before you buy robot pretraining data from a vendor.
Who Collects Robot Pretraining Data?A map of the four categories of who collects robot pretraining data, and how they compare on scale, labels, licensing, and speed.
What Is GEN-0?A plain-language explainer of what GEN-0 is, what it was trained on, and what its official announcement reports about scaling.
What Is Figure's Project Go-Big?A plain-language explainer of what Project Go-Big is, what Figure says it collects and enables, and what it means for teams sourcing humanoid pretraining data.
What Is EgoVerse?A plain-language explainer of what EgoVerse is, what its paper reports, and how it compares with other published egocentric pretraining corpora.
What Is the Egocentric-10K Dataset?A plain-language explainer of what Egocentric-10K is, what its dataset card reports, and how it compares with other published egocentric pretraining corpora.
What Is HumanScale?A plain-language explainer of what HumanScale is and how it fits alongside EgoScale in the research on scaling human video for robot learning.
Pretraining Data for Generalist Robot PoliciesWhat a generalist, multi-task, multi-embodiment robot policy needs from its pretraining data, and why breadth across tasks, scenes and embodiments matters more than raw hours.
Why Stereo Video Matters for Robot Pretraining DataHow a calibrated stereo rig recovers metric depth and richer 3D action labels for robot pretraining, and when it is worth the extra rig complexity.
What Format Should Robot Pretraining Data Be Delivered In?How to choose a delivery format for a large robot pretraining corpus — RLDS, LeRobot, WebDataset, HDF5/zarr, or Rerun .rrd.
Consent and Privacy for Robot Pretraining Data at ScalePretraining scale means many contributors, many bystanders, and many sites — which is exactly where an undocumented consent gap becomes expensive to find later.
How Does Custom Robotics Data Collection Work?A stage-by-stage walkthrough of a custom robotics data collection engagement, from spec to licensed delivery.
How to Scale a Robotics Data Collection From Pilot to ProductionHow a spec pilot turns into a full-volume custom data collection without the process changing between the two.
How to Scope a Custom Robotics Data Collection ProjectThe five decisions to make before a custom robotics data collection starts, and what is stated about the timeline.
How Long Does Custom Robotics Data Collection Take?What determines the timeline of a custom robotics data collection, phase by phase, using only the speeds published on this site.
How to Evaluate a Robot Pretraining Data Pilot Before ScalingWhat to check in a Spec Pilot sample of pretraining data before committing to full volume, and the go / adjust / stop decision that follows.
Robotics Training Data RFP Checklist for AI LabsWhat to ask for, and how to score the answers, when a lab writes an RFP for custom robotics training data.
How Enterprises Procure Custom AI Training DataThe end-to-end procurement path an enterprise follows to buy custom egocentric training data, from internal requirements to accepted delivery.
Why Physical AI Pilots Stall on Training DataWhy the pilot stage of a physical AI deployment is where training data becomes the bottleneck, and how to unblock it.
How to Build a Vetted Workforce for Robotics Data CollectionWhy the people behind a robotics dataset determine its quality, and how a vetted contributor workforce is run.
How to Turn a Human Job Into Robot Training TasksA method for decomposing a real job into task units a robot can be trained on, and the data each needs.
From Demo Data to a Production Robotics Data PipelineWhat separates demo-grade robotics data from a production data pipeline a model team can depend on.
How Global Data Collection Improves Robot GeneralizationWhy geographic and human diversity in robotics data matters, and how a global collection keeps quality constant.
What Should a Robotics Data Collection SLA Include?What to write into the agreement once you have picked a custom robotics data collection provider.
How Is Robot Pretraining Data Annotated?The end-to-end pipeline that turns captured egocentric video into labeled robot pretraining data, from calibration through QA to delivery.
How Much Annotation Does Robot Training Data Need?How much labeling robot training data actually needs, and why the answer is "it depends on the stage and the spec" rather than a fixed number.
How to Budget a Robot Pretraining Data CollectionWhat actually drives the cost of a custom robot pretraining data collection, and how to phase spend from spec to pilot to scale.
Why Does Embodied AI Need Real-World Video?Why embodied AI needs video captured in real environments — what simulation and internet video are missing, and what to require when commissioning real-world capture.
What Is Responsible Egocentric Data Collection?Responsible egocentric collection means a documented consent chain before capture, irreversible de-identification of bystanders, a provenance record that survives delivery, and a buyer-owned licence — each of which a buyer can ask a collection partner to prove rather than assume.
How Are Humanoid Robots Taught Everyday Tasks?From human demonstration to a household task: how training data moves through each stage.
How to Version a Robot Pretraining DatasetA practical guide to tracking spec versions, batches, data cards, and format fields across an ongoing pretraining data collection.
Egocentric Data Collection ChecklistAn ordered, checkbox-style checklist for a custom egocentric program: spec, rig, consent, capture, QA and delivery, with the questions to put to a collection partner at each stage.
How to Evaluate an Egocentric Data SampleA buyer-side, file-level walkthrough of what to check when an egocentric data sample or first batch arrives, and how the result feeds the Spec Pilot decision.
How Are Vision-Language-Action Models Trained?The VLA training pipeline as the listed papers describe it, from a pretrained VLM through pretraining and fine-tuning, and what each stage means for the data you collect.
Can Egocentric Human Video Train VLA Models?Which listed VLA papers use human or egocentric video, at which stage, how they bridge the embodiment gap, and what first-person capture needs to carry.
VLA Dataset Format: What Does a Training Episode Contain?The observation, action and language fields a VLA training episode holds, as RLDS, LeRobot, OpenVLA and Octo document them, and how Firsthand episode fields map onto that structure.
How Much Data Does It Take to Train a VLA Model?VLA dataset size as RT-2, OpenVLA, Octo, π0, GR00T N1 and EgoScale report it, by training stage, with no invented minimum.
How Do You Collect Training Data for a VLA Model?How five published VLA papers say their data was collected, and how a custom egocentric collection is specified, checked and delivered.

Every guide describes the schema in the sample pack.

Download 40 episodes across 4 environments and check the claims against real data.

Get sample data →Read the docs