LVDrive: Latent Visual Representation Enhanced Vision-Language-Action Autonomous Driving Model

TL;DR

LVDrive introduces a latent space future scene prediction task into vision-language-action models, significantly improving autonomous driving performance by enhancing scene understanding and reasoning.

Contribution

It proposes a novel latent space future scene prediction framework with a unified embedding and a two-stage trajectory decoding strategy for autonomous driving.

Findings

LVDrive outperforms existing methods on the Bench2Drive benchmark.

The model achieves significant improvements in closed-loop driving performance.

Joint modeling of future scene and motion prediction enhances reasoning capabilities.

Abstract

Vision-Language-Action (VLA) models have emerged as a promising framework for end-to-end autonomous driving. However, existing VLAs typically rely on sparse action supervision, which underutilizes their powerful scene understanding and reasoning capabilities. Recent attempts to incorporate dense visual supervision via world modeling often overemphasize pixel-level image reconstruction, neglecting semantically meaningful scene representation learning. In this work, we propose LVDrive, a Latent Visual representation enhanced VLA framework for autonomous driving. LVDrive introduces a future scene prediction task into the VLA paradigm, where future representations are learned entirely in a high-level latent space under auxiliary supervision from a pretrained vision backbone. Departing from inefficient autoregressive generation, we jointly model future scene and motion prediction within a unified embedding space, processed in a single forward pass to conduct the future-aware reasoning. We further design a two-stage trajectory decoding strategy that explicitly leverages the learned latent future representations to refine trajectory generation. Extensive experiments on the challenging Bench2Drive benchmark demonstrate that LVDrive achieves significant improvements in closed-loop driving performance, outperforming both action supervised methods and image-reconstruction-based world model approaches.