BlockVLA: Accelerating Autoregressive VLA via Block Diffusion Finetuning

TL;DR

BlockVLA introduces a block diffusion framework that accelerates autoregressive vision-language-action models, enabling faster inference and training efficiency for robotic tasks by combining global causal coherence with local parallel generation.

Contribution

It adapts pretrained autoregressive models into an efficient diffusion policy using block diffusion, reducing inference latency and improving training speed in robotic applications.

Findings

Achieves 3.3× inference acceleration over standard diffusion baselines.

Converges faster in training, especially in complex, long-horizon tasks.

Demonstrates superior performance on LIBERO and SimplerEnv benchmarks.

Abstract

While autoregressive (AR) Vision-Language-Action (VLA) models have demonstrated formidable reasoning capabilities in robotic tasks, their sequential decoding process often incurs high inference latency and may amplify error accumulation during long-horizon execution. Discrete Diffusion Language Models (dLLMs) provide a promising alternative through parallel token refinement, but their practical deployment in robotics remains limited by repeated denoising function evaluations (NFEs) and the difficulty of directly applying standard KV caching to bidirectional iterative decoding. To bridge these paradigms, we propose BlockVLA, a framework that adapts pretrained AR backbones into an efficient discrete diffusion policy through a block diffusion paradigm. BlockVLA maintains autoregressive dependencies at the block level while enabling parallel denoising within each block, thereby combining global causal coherence with local parallel generation. This design enables prefix KV-cache reuse across completed blocks, reduces the effective cost of iterative denoising, and provides a smoother transition from AR pretraining to diffusion-based policy fine-tuning. We conduct extensive evaluations on the LIBERO and SimplerEnv benchmarks. Experimental results demonstrate that our BlockVLA achieves a 3.3$\times$ inference acceleration over standard discrete diffusion baselines. Furthermore, our model exhibits superior training efficiency, with success rates converging substantially faster than baselines, a gain that is particularly pronounced in complex, long-horizon tasks, where BlockVLA achieves significant performance gains in the early stages of training. This work establishes Block Diffusion as a robust bridge between large-scale pretrained AR models and efficient, high-frequency real-time robotic control.