BitVLA: 1-bit Vision-Language-Action Models for Robotics Manipulation

TL;DR

BitVLA introduces a fully native 1-bit vision-language-action model for robotics, significantly reducing memory and latency while maintaining strong task performance, enabling efficient deployment on edge devices.

Contribution

The paper presents BitVLA, a novel 1-bit VLA model built on 1-bit LLM, with a new quantize-then-distill strategy for vision encoder compression, achieving high efficiency without sacrificing accuracy.

Findings

Matches full-precision baseline performance

Reduces model memory by 11.0x

Lowers end-to-end latency by 4.4x

Abstract

Deploying powerful Vision-Language-Action (VLA) models on edge devices is limited by their massive size. In this paper, we take a deployment-oriented view of VLA training: we target efficiency through model design and optimization, rather than relying solely on post-hoc compression. Thus, we propose BitVLA, a fully native 1-bit VLA model for robotic manipulation, where every parameters is ternary, i.e., {-1,0,1}. BitVLA is built on the publicly available 1-bit LLM BitNet b1.58 2B4T, and is trained as a vision-language-action policy that inherits the compactness of 1-bit pretraining while retaining strong task performance. To further reduce the memory footprint of the vision backbone, we introduce Quantize-then-Distill, a post-training quantization-aware training strategy that compresses a full-precision vision encoder to 1.58-bit weights, while a full-precision teacher guides representation alignment during training. Across simulation benchmarks and real-world tasks, BitVLA matches the performance of the full-precision OpenVLA-OFT baseline, while reducing model memory by 11.0x and end-to-end latency by 4.4x. These results suggest a practical path toward training-time efficiency-accuracy co-design for embodied policies, enabling competitive manipulation capability on memory-constrained edge robotic platforms. We release the code in https://github.com/ustcwhy/BitVLA, model weights in https://huggingface.co/lxsy/bitvla-bf16.