How Much Data Does It Take to Train a VLA Model?

There is no single number, and the published papers do not offer one. How much data it takes to train a vision-language-action (VLA) model depends on which stage you mean, which unit the data is counted in, and which robot and tasks the model has to handle. A VLA model takes camera images and a language instruction and outputs an action. Training one usually involves more than one dataset, and each dataset answers a different question.

The π0 paper (https://arxiv.org/abs/2410.24164) is direct about the limit of current knowledge. It says that "it remains unclear how to predict how much and what kind of data is needed to attain near-perfect performance." The same paper adds that its experiments do not yet give a comprehensive understanding of how pre-training datasets should be composed. This guide takes that seriously. It lists what each paper reports about its own data, attributed and linked, and stops short of turning those figures into a rule.

Three handling rules apply to every figure below. Each is quoted in the unit its paper uses: hours, trajectories, demonstrations or timesteps. Units are not converted, summed, averaged or ranked, because the papers do not count data the same way. Where a source does not state a figure, the table says "Not stated in the sources read". Firsthand is not affiliated with any of these authors or labs, and nothing here says a named model was trained, fine-tuned or evaluated on Firsthand data.

• Which stage is this data for? Pretraining, aligned mid-training and task fine-tuning are built to different goals, and the papers report different figures for each.
• Which unit is the target counted in? Hours, episodes, trajectories, demonstrations and timesteps are not interchangeable, so fix one unit in the written spec.
• Which tasks, objects and setting must the model handle? The papers tie fine-tuning data to the task, so list the tasks before you size anything.

The table lists the data each source reports for its own models, by training stage. It covers RT-2, OpenVLA, Octo, π0, GR00T N1 and EgoScale. Octo describes a generalist robot policy rather than a vision-language-action model, and is included because it reports data sizes for both pretraining and fine-tuning.

Read the unit column before the number. The π0 paper itself notes that, because each training example is a timestep, it quantifies data in timesteps in its dataset discussion, while its headline pretraining figure is in hours. RT-2 does not state a robot episode count in the text read for this page; it refers to the earlier RT-1 dataset instead. A separate guide on this site covers pretraining corpora for robot foundation models in general, including corpora outside these six papers. This guide is limited to what the six papers report.

Because they have different jobs, and the papers say so. The π0 paper writes that "the requirements for the pre-training and post-training datasets are distinct": the pre-training dataset "should cover as many tasks as possible, and within each of those tasks should cover a diversity of behaviors", while the post-training dataset should cover behaviors that "exhibit a consistent and fluent strategy". Its stated reasoning is that diverse, lower-quality data helps the model recover from mistakes, and the high-quality data teaches it to perform the task well (https://arxiv.org/abs/2410.24164).

EgoScale describes a different split. It reports a two-stage transfer recipe, large-scale human pretraining followed by "lightweight aligned human–robot mid-training", and then post-training on downstream tasks (https://arxiv.org/abs/2602.16710). RT-2 describes co-fine-tuning, where robot data stays mixed with Internet-scale vision-language data instead of replacing it (https://arxiv.org/abs/2307.15818). GR00T N1 arranges its pre-training data as a pyramid of real-robot, synthetic and human video sources (https://arxiv.org/abs/2503.14734).

The practical reading is that amounts differ because goals differ, not because one stage is bigger than another by a fixed ratio. No source read for this page gives a ratio between stages, so this guide does not either. The second table maps each stage to what its data has to contain, what the Firsthand site states it delivers, and the field in the existing episode schema where one exists.

The language row is deliberately narrow. The only language data the site ties to Firsthand capture is one task instruction per episode, kept on the same clock as the video and the action fields. Firsthand captures first-person human video, not robot or teleoperation data, so any robot-side data your recipe needs has to come from somewhere else.

Several papers treat diversity as part of the answer to "how much", and none of them reduces it to a single exchange rate. The π0 paper says its pre-training data should cover "a diversity of behaviors" within each task. It also warns that its task count is not comparable with other papers: its own definition of a task is broad (the bussing task covers many dishes, cups and utensils), so "the actual range of behaviors in our dataset is significantly broader than this number of “tasks” would imply" (https://arxiv.org/abs/2410.24164). That is a reason not to compare task counts across the rows of the first table.

EgoScale gives the most explicit data-size study among the six papers. It pretrains models with 1k, 2k, 4k, 10k and 20k hours of human data. It reports that smaller datasets (1k to 2k hours) "initially reduce validation loss but later plateau or degrade, indicating overfitting to limited behavioral diversity", while larger datasets (10k to 20k hours) show stable, monotonic improvement. It also reports a log-linear relationship between hours of human data and validation loss, and no signs of saturation in the range it explored (https://arxiv.org/abs/2602.16710). Those are findings about EgoScale’s own data and model, in the range tested. They are not a rule for another dataset.

RT-2 points the same way from the robot side. Its limitations section says the robot does not acquire the ability to perform new motions from web-scale pretraining, and attributes this to "the dataset not being varied enough along the axes of skills" (https://arxiv.org/abs/2307.15818).

The π0 paper leaves the composition question open. It says that understanding what type of data is more helpful to add, and how it should be weighted, "remains an open problem". A practical way to size a collection is therefore to list the scenes, tasks, objects and behaviors the model must cover, then count hours or episodes against that list. Fixing a total first and hoping it covers everything gets the order backwards.

Two of the six papers use egocentric human video, and both report on their own data. EgoScale trains a vision-language-action model on 20,854 hours of action-labeled egocentric human video, then adds a "lightweight" aligned human-robot mid-training stage. The mid-training set it describes is about 50 hours of human data and 4 hours of robot data across 344 tabletop tasks. Its abstract says the recipe enables "one-shot task adaptation with minimal robot supervision" (https://arxiv.org/abs/2602.16710).

GR00T N1 places human egocentric video from open datasets at the base of its data pyramid, together with web data, and builds up through synthetic data to real-robot data at the top. It also observes that robot data "scales linearly with human labor", because a human operator typically has to teleoperate the robot to produce each trajectory (https://arxiv.org/abs/2503.14734). RT-2 lists "videos of humans" as a direction for future work on acquiring new skills (https://arxiv.org/abs/2307.15818).

The caution is about actions. Human video has no robot actions, so someone has to decide how hand motion maps to the actions your model learns. Firsthand delivers the human side: 21-joint 3D hand pose with a per-joint visibility flag, wrist trajectories, contact events, action segments, a 200 Hz IMU and stereo depth where the spec calls for it, all on one hardware clock. Whether that is enough for your model, or has to be paired with robot-side demonstrations, is a decision for the written spec and a pilot batch, not a number this page can supply.

The sources report what they used, not what you need. OpenVLA fine-tunes on "small datasets with 10–150 demonstrations of a target task" across seven tasks, with 10, 45, 50, 70 and 150 demonstrations for individual tasks (https://arxiv.org/abs/2406.09246). Octo uses "around 100 trajectories" in all of its finetuning experiments (https://arxiv.org/abs/2405.12213). π0 describes post-training sets running from 5 hours for the simplest tasks to 100 or more hours for the most complex (https://arxiv.org/abs/2410.24164). GR00T N1 reports operators collecting task-specific post-training data for 15 minutes to 3 hours depending on task complexity, then filtering out low-quality trajectories (https://arxiv.org/abs/2503.14734).

What those figures share is the unit problem: demonstrations, trajectories, hours and collection time are four different counts, and none of the papers claims a minimum. The stated driver is the task. π0 says different tasks "require very different datasets". OpenVLA separates narrow single-instruction tasks from multi-instruction tasks in which the target is specified in the language instruction. GR00T N1 ties collection time to task complexity.

To put a number on your own task, write the task list first, then run a small first batch. Firsthand calls that first batch a Spec Pilot: a small, fast batch that proves the spec and accept criteria on real footage before volume is added. Where coverage is deep, the first validated batch lands in a median of 48 hours from a signed spec. That is a turnaround figure for the first batch. It is not a statement about how much data a VLA needs.

Public mixtures such as Open X-Embodiment cover the robots and scenes their publishers collected. Custom VLA data collection makes sense when your tasks, objects, environments or label vocabulary are not in that coverage, or when you need a narrow, filtered fine-tuning set faster than your own team can capture one.

• Your target tasks or objects are missing from the public data you can access, and a pilot batch would show it quickly.
• You need human-side egocentric episodes with hand pose, contact events and action segments on one clock, and can map hand motion to your own action space.
• You want the unit, the accept criteria and the delivery format written down before capture starts, so the dataset loads into the training code you already run.
• You need to ship on data you own, under a perpetual commercial license, instead of data with a restricted license.

Firsthand collects custom first-person data on demand, to a written spec. Each episode carries one task instruction, 21-joint 3D hand pose with a per-joint visibility flag, wrist trajectories, contact events and action segments, with a 200 Hz IMU and stereo depth where the spec calls for them. Delivery is in RLDS, LeRobot, WebDataset or HDF5 under a perpetual, buyer-owned commercial license. The first validated batch lands in a median of 48 hours where coverage is deep, and in three to four weeks where contributors have to be recruited first. The custom dataset request below starts the written spec.