What Is the FAST Action Tokenizer?

FAST is introduced in FAST: Efficient Action Tokenization for Vision-Language-Action Models (arXiv 2501.09747) by Karl Pertsch and co-authors, with an official project page, a Hugging Face repository and the openpi code. The paper’s starting point is that autoregressive VLA models must choose how continuous actions become discrete tokens, and that simple per-dimension, per-timestep binning typically performs poorly when learning dexterous skills from high-frequency robot data. The project page adds that binning is passable for simple behaviors but breaks down for precise, high-frequency control, and that diffusion or flow matching performs better but takes much longer to train. FAST is the authors’ answer: compress the action chunk before training on it, so the model predicts fewer, denser tokens. This page is a definition, not a guide to running the code.

FAST is third-party research. Firsthand is not affiliated with its authors or with Physical Intelligence, and makes no claim that FAST, pi0-FAST or any other named model was trained, fine-tuned or evaluated on Firsthand data. Firsthand captures first-person human video, collected fast, on demand and to a written spec, not robot or teleoperation data, and it does not ship tokenized actions. The raw fields the site states are 3D hand pose at 21 keypoints per hand with a per-joint visibility flag, action segments, and 6-DoF IMUs at 200 Hz on one clock. The only language field is one task instruction per episode. Whether the FAST tokenizer suits human hand-pose data is Not stated in the sources read.

The project page shows five steps. A normalized action chunk goes in. A discrete cosine transform (DCT) turns each action dimension into frequency components. The components are quantized, leaving a sparse frequency matrix with one row per dimension. The matrix is flattened with low-frequency components first. Byte pair encoding (BPE) then compresses the flattened sequence into dense action tokens. The authors describe the DCT as a signal-compression technique used in JPEG and MP3 codecs, and BPE as a compression algorithm often used to train language models.

The Hugging Face card recommends applying the tokenizer to 1-second action chunks that have been pre-normalized to a range of -1 to 1, using quantile normalization. Decoding has to map tokens back into a [time_horizon, action_dim] matrix, so those dimensions are either saved on the first forward call or passed to decode(). The vocabulary size and the quantization scale are Not stated in the sources read.

Under binning, the project page says, each dimension of each action step is represented with a discrete bin, so the token count follows the chunk length and the number of dimensions. FAST compresses the whole chunk first. The page reports that it condenses a chunk into typically 30 to 60 tokens, a 10x compression over prior action tokenization approaches, for action chunks that range from 3 to 5 actions on crude systems to 20 to 50 on high-frequency dexterous robots.

The authors also report that in their comparisons standard discretization methods could not solve any of the dexterous tasks in their experimental repertoire, while FAST could. These are the authors’ own results on their own tasks. The binning settings behind the 10x figure are Not stated in the sources read.

pi0-FAST is the first autoregressive generalist policy the authors trained with FAST. It builds on the pi0 model and uses the same backbone and training dataset. The project page says it solves the same complex, dexterous tasks as the diffusion-based pi0 while training 5x faster, and the abstract says FAST combined with pi0 scales to 10k hours of robot data and matches diffusion VLAs while reducing training time by up to 5x.

The project page names one shortcoming: autoregressive decoding in pi0-FAST is significantly slower than the flow-matching decoding in pi0. The openpi repository lists pi0-FAST as an autoregressive VLA based on the FAST tokenizer, with a base checkpoint for fine-tuning and an Apache License 2.0. Its PyTorch support does not currently include pi0-FAST.

FAST+ is the universal tokenizer the authors released, trained on 1M real robot action trajectories. The abstract says it can be used as a black-box tokenizer for robot action sequences with diverse action spaces and control frequencies. The Hugging Face card loads it as a transformers AutoProcessor, uses scipy for the DCT, and provides a fit() function that trains a new tokenizer on your own action chunks, which it says should typically take seconds to minutes. The project page also reports that FAST let the authors train a generalist policy on the DROID dataset that followed language commands in new environments.

FAST tokenizes continuous action chunks, so an episode used this way needs a time-aligned continuous action signal, plus observations and, for a language-conditioned policy, an instruction. Firsthand does not deliver robot action trajectories. The table below sets those needs against the raw fields the site states, and whether they can become action targets is a modeling decision that stays with your pipeline.

If your program needs different fields, signals or environments, collection is custom, fast and on demand, scoped by a written spec. Settle the fields, the delivery format and the wording of the single task instruction per episode on a Spec Pilot, a small first batch, then request a custom dataset.

What does FAST stand for?

Frequency-space Action Sequence Tokenization. It is a compression-based tokenization scheme for robot actions, built on the discrete cosine transform, according to the paper abstract.

Is FAST the same as pi0-FAST?

No. FAST is the tokenizer. pi0-FAST is an autoregressive policy that builds on pi0 and uses FAST to turn actions into tokens.

Why not use diffusion or flow matching instead?

The project page says diffusion and flow matching perform well but take much longer to train. FAST lets an autoregressive model train up to 5x faster at similar performance, but its decoding is significantly slower than pi0’s flow-matching decoding.

Can FAST be trained on my own action data?

Per the Hugging Face card, yes for robot action chunks: its fit() function trains a new tokenizer on your own data. Use on human hand-pose data is Not stated in the sources read.

Does Firsthand deliver FAST tokens?

No. Firsthand delivers raw fields from first-person human capture, such as 3D hand pose, action segments and IMU, and does not ship tokenized actions. Tokenizing them would be a step in your own pipeline.

What language data does Firsthand provide?

One task instruction per episode, paired with the video and sensor streams. The site states no narration or per-step language labels.