What Is OpenVLA?

OpenVLA is introduced in the paper OpenVLA: An Open-Source Vision-Language-Action Model (arXiv 2406.09246), with an official project page and GitHub repository. The authors argue that fine-tuning vision-language-action (VLA) models is a way to teach robots new skills, but that existing VLAs were largely closed and that efficient fine-tuning for new tasks was unexplored. OpenVLA is their open answer to both.

OpenVLA is third-party research, and Firsthand is not part of it, endorsed by it or affiliated with its authors. Firsthand makes no claim that its data trained or was used to evaluate OpenVLA, and no performance claim against it. In the sources, the training data is robot trajectories from Open X-Embodiment, and egocentric human video is not described as part of the mix. What Firsthand states is custom egocentric collection for VLA-style training: synchronized egocentric video, a paired task instruction and the action that carried it out (3D hand pose, gripper state or a robot action log), hardware-triggered onto one clock, to a written spec, on demand.

The authors fine-tune a pretrained Prismatic-7B vision-language model. It has three parts: a fused visual encoder (SigLIP and DINOv2) that turns images into patch embeddings, a projector that maps them into the language model’s input space, and a Llama 2 7B backbone that predicts tokenized actions. The tokens are decoded into continuous actions that the robot can execute directly.

The authors curate 970k robot manipulation trajectories from the Open X-Embodiment dataset, spanning many tasks, scenes and robot embodiments, and train on 64 A100 GPUs for 15 days. Per the project page, the vision-language model is fine-tuned solely on robot action data, not co-fine-tuned with internet data.

The paper reports that OpenVLA outperforms RT-2-X, a closed 55B model, by 16.5% in absolute task success across 29 tasks and multiple robot embodiments, with 7x fewer parameters. It also reports a 20.4% gain over from-scratch imitation methods such as Diffusion Policy in fine-tuning.

The project page adds that RT-2-X did better on difficult semantic generalization tasks and that Diffusion Policy was strongest on narrower, more precise tasks. These are the authors’ results on their own robots and tasks, and Firsthand draws no comparison from them.

All three are described as vision-language-action style models, but they differ in how they write actions, how big they are and what they train on. The table below sets out only what each entry’s official sources state, and Firsthand ranks none of them.

The authors report that LoRA matched full fine-tuning while updating only 1.4% of the parameters, and the paper states it can be fine-tuned on consumer GPUs. The paper says the authors release checkpoints, fine-tuning notebooks and a PyTorch codebase.

The README says all code is under the MIT License, but the pretrained models derive from Llama 2 and are subject to the Llama Community License. Training data licences are Not stated in the sources read.

The paper credits its results to added data diversity and new model components, and the project page shows the full model behaving more reliably than a version without Open X-Embodiment pretraining. The README says a custom dataset outside Open X-Embodiment should be converted to RLDS, which Firsthand lists among its delivery formats.

Firsthand makes no claim that its data fine-tunes OpenVLA, and mapping to OpenVLA’s action space is Not stated.

Write the target skills, environments and signals into a skill spec and confirm them on a Spec Pilot, a small fast first batch. This is on-demand custom collection, not a catalog download.

Where coverage is deep, the median from signed spec to first validated batch is 48 hours; where it is thin, three to four weeks. Billing is per validated hour.

What is OpenVLA?

An open-source vision-language-action model with 7 billion parameters, built on a Llama 2 language model and a fused DINOv2 and SigLIP visual encoder, for controlling robots.

What data was OpenVLA trained on?

970k robot manipulation trajectories from the Open X-Embodiment dataset, spanning many tasks, scenes and robot embodiments.

Is OpenVLA open source?

The code is under the MIT License and checkpoints are on Hugging Face. The models derive from Llama 2 and are subject to the Llama Community License.

Is OpenVLA the same as RT-2 or π0?

No. They are separate models from different authors that write robot actions in different ways. See the side-by-side table for what each one’s sources state.

Was OpenVLA trained on egocentric human video?

The sources describe robot trajectories from Open X-Embodiment. Egocentric human video is not described as part of its training mix in the sources read.

Is Firsthand data part of OpenVLA?

No. Firsthand is not part of it, endorsed by it or affiliated with its authors, and makes no claim that its data trained or evaluated OpenVLA.