FUTURE-VLA: Forecasting Unified Trajectories Under Real-time Execution

TL;DR

FUTURE-VLA introduces a real-time, unified vision-language model for long-horizon control and future forecasting in robotics, achieving state-of-the-art success rates with minimal latency by using adaptive compression and latent autoregression.

Contribution

It presents a novel architecture that reformulates long-horizon control as a sequence-generation task, enabling efficient, real-time spatiotemporal reasoning for robotics applications.

Findings

Achieves 99.2% success on LIBERO

Attains 75.4% success on RoboTwin

Maintains latency comparable to single-frame models

Abstract

General vision-language models increasingly support unified spatiotemporal reasoning over long video streams, yet deploying such capabilities on robots remains constrained by the prohibitive latency of processing long-horizon histories and generating high-dimensional future predictions. To bridge this gap, we present FUTURE-VLA, a unified architecture that reformulates long-horizon control and future forecasting as a monolithic sequence-generation task. Adopting a dual-sided efficiency paradigm, FUTURE-VLA leverages a temporally adaptive compression strategy to maximize spatiotemporal information density, enabling the ingestion of extensive multi-view histories while maintaining constant inference latency. Simultaneously, it performs latent-space autoregression to align actionable dynamics with reviewable visual look-aheads in a single forward pass. These real-time predictive capabilities further enable a prediction-guided Human-In-the-Loop mechanism via interactive execution gating, allowing operators to dynamically validate behaviors based on interpretable future previews. Extensive evaluations demonstrate that FUTURE-VLA establishes new state-of-the-art performance, attaining success rates of 99.2% on LIBERO, 75.4% on RoboTwin, and 78.0% on a real-world Piper platform, all with a $16\times$ extended spatiotemporal window while maintaining the inference latency of a single-frame baseline.