Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance

TL;DR

This paper introduces Spec-VLA, a speculative decoding framework that accelerates vision-language-action models by relaxing acceptance criteria, achieving significant speedup without loss of accuracy.

Contribution

The paper proposes a novel speculative decoding method with relaxed acceptance for VLA models, enabling faster generation while maintaining performance.

Findings

Achieves 1.42x speedup over baseline

Increases acceptance length by 44%

Maintains success rate with faster decoding

Abstract

Vision-Language-Action (VLA) models have made substantial progress by leveraging the robust capabilities of Visual Language Models (VLMs). However, VLMs' significant parameter size and autoregressive (AR) decoding nature impose considerable computational demands on VLA models. While Speculative Decoding (SD) has shown efficacy in accelerating Large Language Models (LLMs) by incorporating efficient drafting and parallel verification, allowing multiple tokens to be generated in one forward pass, its application to VLA models remains unexplored. This work introduces Spec-VLA, an SD framework designed to accelerate VLA models. Due to the difficulty of the action prediction task and the greedy decoding mechanism of the VLA models, the direct application of the advanced SD framework to the VLA prediction task yields a minor speed improvement. To boost the generation speed, we propose an effective mechanism to relax acceptance utilizing the relative distances represented by the action tokens of the VLA model. Empirical results across diverse test scenarios affirm the effectiveness of the Spec-VLA framework, and further analysis substantiates the impact of our proposed strategies, which enhance the acceptance length by 44%, achieving 1.42 times speedup compared with the OpenVLA baseline, without compromising the success rate. The success of the Spec-VLA framework highlights the potential for broader application of speculative execution in VLA prediction scenarios.