VQ-VLA: Improving Vision-Language-Action Models via Scaling Vector-Quantized Action Tokenizers

TL;DR

This paper presents a scalable vector quantization-based action tokenizer trained on an extensive dataset, significantly improving the efficiency and coherence of action generation in vision-language-action models, especially in real-world robotic tasks.

Contribution

Introduces a novel, large-scale vector quantized action tokenizer that leverages synthetic data, enabling zero-shot adaptation and improved performance in downstream tasks.

Findings

Tokenizer achieves up to 30% higher success rate on real-world tasks.

Synthetic data has minimal domain gap with real trajectories.

Model accelerates inference and produces smoother actions.

Abstract

In this paper, we introduce an innovative vector quantization based action tokenizer built upon the largest-scale action trajectory dataset to date, leveraging over 100 times more data than previous approaches. This extensive dataset enables our tokenizer to capture rich spatiotemporal dynamics, resulting in a model that not only accelerates inference but also generates smoother and more coherent action outputs. Once trained, the tokenizer can be seamlessly adapted to a wide range of downstream tasks in a zero-shot manner, from short-horizon reactive behaviors to long-horizon planning. A key finding of our work is that the domain gap between synthetic and real action trajectories is marginal, allowing us to effectively utilize a vast amount of synthetic data during training without compromising real-world performance. To validate our approach, we conducted extensive experiments in both simulated environments and on real robotic platforms. The results demonstrate that as the volume of synthetic trajectory data increases, the performance of our tokenizer on downstream tasks improves significantly-most notably, achieving up to a 30% higher success rate on two real-world tasks in long-horizon scenarios. These findings highlight the potential of our action tokenizer as a robust and scalable solution for real-time embodied intelligence systems, paving the way for more efficient and reliable robotic control in diverse application domains.Project website: https://xiaoxiao0406.github.io/vqvla.github.io