OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

TL;DR

This paper introduces OneDrive, a unified vision-language-action model for autonomous driving that leverages a single transformer decoder to handle multiple tasks, achieving state-of-the-art results and efficient inference.

Contribution

The work presents a novel unified framework that integrates heterogeneous driving tasks within a pretrained VLM using a single causal decoder, enhancing efficiency and performance.

Findings

Achieves 0.28 L2 and 0.18 collision rate on nuScenes.

Attains 86.8 PDMS on NAVSIM.

Reduces inference latency by approximately 40%.

Abstract

Vision-Language Models(VLMs) excel at autoregressive text generation, yet end-to-end autonomous driving requires multi-task learning with structured outputs and heterogeneous decoding behaviors, such as autoregressive language generation, parallel object detection and trajectory regression. To accommodate these differences, existing systems typically introduce separate or cascaded decoders, resulting in architectural fragmentation and limited backbone reuse. In this work, we present a unified autonomous driving framework built upon a pretrained VLM, where heterogeneous decoding behaviors are reconciled within a single transformer decoder. We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling. By organizing visual and structured query tokens within a single causal decoder, structured queries can naturally condition on visual context through the original attention mechanism. Textual and structured outputs share a common attention backbone, enabling stable joint optimization across heterogeneous tasks. Trajectory planning is realized within the same causal LLM decoder by introducing structured trajectory queries. This unified formulation enables planning to share the pretrained attention backbone with images and perception tokens. Extensive experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance, including 0.28 L2 and 0.18 collision rate on nuScenes open-loop evaluation and competitive results (86.8 PDMS) on NAVSIM closed-loop evaluation. The full model preserves multi-modal generation capability, while an efficient inference mode achieves approximately 40% lower latency. Code and models are available at https://github.com/Z1zyw/OneDrive