OpenREAD: Reinforced Open-Ended Reasoning for End-to-End Autonomous Driving with LLM-as-Critic

TL;DR

OpenREAD introduces an end-to-end reinforcement fine-tuning framework using large language models as critics, significantly improving reasoning and planning in autonomous driving by leveraging open-ended reasoning and large-scale annotations.

Contribution

The paper presents a novel open-ended reasoning framework for autonomous driving that integrates large language models as critics in reinforcement fine-tuning, enhancing reasoning and planning capabilities.

Findings

Achieves state-of-the-art performance on reasoning benchmarks.

Improves both upstream and downstream autonomous driving tasks.

Effectively utilizes large-scale Chain-of-Thought annotations.

Abstract

Recently, two-stage fine-tuning strategies, e.g., acquiring essential driving knowledge through supervised fine-tuning (SFT) and further enhancing decision-making and planning via reinforcement fine-tuning (RFT), have shown strong potential in advancing the knowledge-driven autonomous driving (AD) paradigm. However, the learning nature of SFT still limits the generalization of reasoning, thereby constraining the full potential of driving performance. Meanwhile, current RFT approaches are primarily applied to downstream tasks, since scene understanding is an open-ended problem where corresponding rewards are difficult to quantify. To address these limitations, we propose OpenREAD, an OPEN-ended REasoning reinforced vision-language model (VLM)-based autonomous driving (AD) framework that enables end-to-end RFT across the full spectrum from high-level reasoning to low-level trajectory planning. Specifically, we begin by constructing large-scale Chain-of-Thought (CoT) annotations on open-source driving-related knowledge datasets, and employ the powerful Qwen3 large language model (LLM) as the critic in RFT to quantify reasoning quality for open-ended questions during reward modeling. Extensive experiments confirm that joint end-to-end RFT yields substantial improvements in both upstream and downstream tasks, enabling OpenREAD to achieve state-of-the-art performance on reasoning and planning benchmarks.