FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Model

TL;DR

FutureX introduces a Chain-of-Thought approach with a latent world model to improve end-to-end autonomous driving planning, enabling better scene reasoning and trajectory refinement in dynamic environments.

Contribution

It presents a novel CoT-driven pipeline that dynamically switches between reasoning and instant modes, significantly enhancing motion planning accuracy and safety.

Findings

Achieves 6.2 PDMS improvement on NAVSIM

Produces more rational motion plans

Reduces collision rates

Abstract

In autonomous driving, end-to-end planners learn scene representations from raw sensor data and utilize them to generate a motion plan or control actions. However, exclusive reliance on the current scene for motion planning may result in suboptimal responses in highly dynamic traffic environments where ego actions further alter the future scene. To model the evolution of future scenes, we leverage the World Model to represent how the ego vehicle and its environment interact and change over time, which entails complex reasoning. The Chain of Thought (CoT) offers a promising solution by forecasting a sequence of future thoughts that subsequently guide trajectory refinement. In this paper, we propose FutureX, a CoT-driven pipeline that enhances end-to-end planners to perform complex motion planning via future scene latent reasoning and trajectory refinement. Specifically, the Auto-think Switch examines the current scene and decides whether additional reasoning is required to yield a higher-quality motion plan. Once FutureX enters the Thinking mode, the Latent World Model conducts a CoT-guided rollout to predict future scene representation, enabling the Summarizer Module to further refine the motion plan. Otherwise, FutureX operates in an Instant mode to generate motion plans in a forward pass for relatively simple scenes. Extensive experiments demonstrate that FutureX enhances existing methods by producing more rational motion plans and fewer collisions without compromising efficiency, thereby achieving substantial overall performance gains, e.g., 6.2 PDMS improvement for TransFuser on NAVSIM. Code will be released.