What Is WorldVLA?

WorldVLA is introduced in WorldVLA: Towards Autoregressive Action World Model (arXiv 2506.21539), first submitted on 26 June 2025 by Jun Cen and colleagues at DAMO Academy (Alibaba Group), Hupan Lab and Zhejiang University. The abstract describes an autoregressive action world model that unifies action and image understanding and generation, integrating a vision-language-action (VLA) model and a world model in one framework. The world model predicts future images from both action and image understanding, with the stated purpose of learning the underlying physics of the environment to improve action generation. The action model generates the next actions from image observations, which the authors say aids visual understanding and in turn helps the world model’s image generation. The paper’s Table 1 separates action models, video prediction models, world models and action world models by their inputs and outputs. This page draws only on the paper, the official GitHub repository and the Hugging Face model card; no separate project page was linked in the sources read.

The paper formulates two components. The action model produces an action from a history of image observations and a language instruction. The world model predicts the next frame from a history of earlier observations and the actions taken. WorldVLA is one model that does both, and it uses three tokenizers, for images, text and actions, whose tokens share one vocabulary so that understanding and generation across all three can sit in a single language-model architecture. The authors give three reasons for mixing world model data into training: the world model learns environmental physics that helps manipulation, it lets the system simulate and evaluate candidate actions, and it requires a precise reading of action inputs that in turn supports the action model. In the other direction, the action model improves the visual understanding the world model relies on. The world model prompt needs no task instruction, because the action determines the next state.

Per the paper, the model is initialized from Chameleon, a unified image understanding and generation model. The image tokenizer is a VQ-GAN with a compression ratio of 16 and a codebook of 8192, producing 256 tokens for a 256 x 256 image and 1024 tokens for a 512 x 512 image. The action tokenizer splits each dimension of a continuous robot action into 256 bins, and an action is 7 tokens: 3 relative positions, 3 relative angles and 1 absolute gripper state. The text tokenizer is a BPE tokenizer with a vocabulary of 65,536 that includes the 8192 image tokens and 256 action tokens.

Action model data and world model data are mixed. The loss is the action cross-entropy plus alpha times the world model cross-entropy, with alpha fixed at 0.04 because image tokens far outnumber action tokens. The default input is two images, the action chunk is 10 actions for LIBERO Long and 5 for the other three suites, and the world model runs a single round. The paper does not state a parameter count, the training hardware or the training time in the sections read, so those are Not stated in the sources read.

The authors found that generating several actions in sequence with a standard causal mask degrades an autoregressive action model. They attribute this to limited action generalization in a backbone pretrained mostly on images and text, so errors in earlier actions propagate to later ones. Their fix is a mask under which the current action relies only on text and visual inputs, which lets the model generate a chunk of actions in parallel. The world model part keeps the causal mask.

The introduction reports that conventional autoregressive chunk generation lowers the grasping success rate by 10% to 50%, and that the mask gives a 4% to 23% improvement. In the paper’s ablation, chunking without the mask averages 54.0, and chunking with the mask averages 76.6.

Experiments use the LIBERO benchmark (Spatial, Object, Goal, Long and LIBERO-90). The authors filter out unsuccessful trajectories and no-operation actions, then split 90% of trajectories for training and 10% for validation, except in the main comparison table, which uses all available data. Action models are scored by success rate over 50 rollouts per task. The world model is scored on the validation set with FVD, PSNR, SSIM and LPIPS.

Without robot-data pretraining, WorldVLA averages 79.1 at 256 x 256 and 81.8 at 512 x 512, against 76.5 for the pretrained discrete OpenVLA baseline. In the ablations, adding the world model raises the action model’s average from 62.8 to 67.2. For 50-frame generation, the action world model lowers FVD from 718.6 to 674.1, while at 10 frames its FVD is slightly higher (255.1 versus 250.0).

The GitHub README dates the WorldVLA release of models, training code and evaluation code on LIBERO to 2025.06.23. A later entry, dated 2025.11.10, says WorldVLA was upgraded to RynnVLA-002, and the repository is now titled RynnVLA-002. This page does not report RynnVLA-002 results. The Hugging Face model card lists 256 x 256 and 512 x 512 checkpoints for each LIBERO suite, with success rates matching the paper’s table.

The repository README says assets and code are under Apache 2.0 unless specified otherwise. The Hugging Face card text read and the paper state no licence for the checkpoints, so that is Not stated in the sources read. This is not legal advice.

They are different kinds of data. WorldVLA trains on paired images and robot actions from the LIBERO benchmark, with actions encoded as relative position, relative angle and gripper tokens. Firsthand captures real human egocentric demonstrations from a head-mounted camera and two wrist cameras. Nothing on this site describes robot action or joint data, action tokens, teleoperation data, simulation data or video captions, and none of that is claimed here.

Firsthand also makes no claim that its data is world-model training data or carries future-frame or action-conditioned video labels. It makes no claim that its data trained, was used with or is compatible with WorldVLA, RynnVLA-002, LIBERO or any model named here. The site’s general view is in its guide on egocentric video for world models, linked below.

Firsthand collects real egocentric demonstrations on demand, to a written spec, rather than selling a fixed catalog. The rig is a head-mounted global-shutter camera plus two wrist cameras, with active-stereo depth and 3x IMU on one hardware-triggered PTP clock. Episodes carry 3D hand pose at 21 joints per hand with a per-joint visibility flag, wrist trajectories, contact events, frame-accurate action segments and one task instruction per episode, which is the only language field the site states.

You write the spec, confirm it on a Spec Pilot, and pay per validated hour. The site states a 48-hour median turnaround where coverage is deep and three to four weeks where it is thin. Delivery is RLDS, LeRobot, WebDataset or HDF5, with a buyer-owned commercial license. Request a custom dataset below.

What is WorldVLA?

An autoregressive action world model that unifies action and image understanding and generation, integrating a VLA model and a world model in one framework. It is introduced in arXiv 2506.21539.

What is an action world model?

In the paper’s Table 1, an action world model takes text, video and actions as input and outputs both video and actions. A plain world model outputs video only, and an action model outputs actions only.

What does the action attention mask do?

It stops the current action from attending to earlier actions, so each action depends only on the text and visual inputs. The authors say this limits error propagation when generating action chunks.

Is WorldVLA the same as RynnVLA-002?

No. The repository README says WorldVLA was upgraded to RynnVLA-002, which adds a continuous Action Transformer, wrist camera input and generation, and state input. This page covers WorldVLA as described in its paper.

Can WorldVLA be used commercially?

The repository README says assets and code are under Apache 2.0 unless specified otherwise. The Hugging Face card text read and the paper state no licence for the checkpoints, so that is Not stated in the sources read. Check the repositories directly; this is not legal advice.

Does Firsthand use WorldVLA, and can I request a custom dataset?

Firsthand is not affiliated with WorldVLA’s authors and makes no claim of use, training or compatibility. Firsthand data is real human egocentric capture. You can request a custom egocentric dataset: write a spec, confirm it on a Spec Pilot, pay per validated hour.