FAST: Efficient Action Tokenization for Vision-Language-Action Models

TL;DR

This paper introduces FAST, a novel frequency-based action tokenization method for vision-language-action models, enabling efficient learning of dexterous, high-frequency robotic behaviors and significantly reducing training time.

Contribution

We propose FAST, a compression-based tokenization scheme using discrete cosine transform, and release FAST+ as a universal robot action tokenizer trained on extensive real robot data.

Findings

FAST enables training on high-frequency, dexterous tasks where standard methods fail.

Using FAST+ improves training efficiency, reducing time by up to 5x.

Our approach matches diffusion VLA performance on large-scale robot data.

Abstract

Autoregressive sequence models, such as Transformer-based vision-language action (VLA) policies, can be tremendously effective for capturing complex and generalizable robotic behaviors. However, such models require us to choose a tokenization of our continuous action signals, which determines how the discrete symbols predicted by the model map to continuous robot actions. We find that current approaches for robot action tokenization, based on simple per-dimension, per-timestep binning schemes, typically perform poorly when learning dexterous skills from high-frequency robot data. To address this challenge, we propose a new compression-based tokenization scheme for robot actions, based on the discrete cosine transform. Our tokenization approach, Frequency-space Action Sequence Tokenization (FAST), enables us to train autoregressive VLAs for highly dexterous and high-frequency tasks where standard discretization methods fail completely. Based on FAST, we release FAST+, a universal robot action tokenizer, trained on 1M real robot action trajectories. It can be used as a black-box tokenizer for a wide range of robot action sequences, with diverse action spaces and control frequencies. Finally, we show that, when combined with the pi0 VLA, our method can scale to training on 10k hours of robot data and match the performance of diffusion VLAs, while reducing training time by up to 5x.