Compressor-VLA: Instruction-Guided Visual Token Compression for Efficient Robotic Manipulation

TL;DR

Compressor-VLA introduces an instruction-guided visual token compression framework that significantly reduces computational costs in embodied AI models while maintaining task performance, enabling efficient real-time robotic manipulation.

Contribution

It presents a novel hybrid, instruction-conditioned token compression method combining holistic and spatial detail preservation for VLA models.

Findings

Achieves 59% reduction in FLOPs and over 3x fewer visual tokens.

Maintains competitive success rates on LIBERO benchmark.

Demonstrates effective sim-to-real transfer on dual-arm robots.

Abstract

Vision-Language-Action (VLA) models have emerged as a powerful paradigm in Embodied AI. However, the significant computational overhead of processing redundant visual tokens remains a critical bottleneck for real-time robotic deployment. While standard token pruning techniques can alleviate this, these task-agnostic methods struggle to preserve task-critical visual information. To address this challenge, simultaneously preserving both the holistic context and fine-grained details for precise action, we propose Compressor-VLA, a novel hybrid instruction-conditioned token compression framework designed for efficient, task-oriented compression of visual information in VLA models. The proposed Compressor-VLA framework consists of two token compression modules: a Semantic Task Compressor (STC) that distills holistic, task-relevant context, and a Spatial Refinement Compressor (SRC) that preserves fine-grained spatial details. This compression is dynamically modulated by the natural language instruction, allowing for the adaptive condensation of task-relevant visual information. Experimentally, extensive evaluations demonstrate that Compressor-VLA achieves a competitive success rate on the LIBERO benchmark while reducing FLOPs by 59% and the visual token count by over 3x compared to its baseline. The real-robot deployments on a dual-arm robot platform validate the model's sim-to-real transferability and practical applicability. Moreover, qualitative analyses reveal that our instruction guidance dynamically steers the model's perceptual focus toward task-relevant objects, thereby validating the effectiveness of our approach.