WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving

TL;DR

WorldRFT introduces a planning-focused latent world model for autonomous driving, integrating hierarchical planning, local refinement, and reinforcement fine-tuning to improve safety and performance without relying on perception annotations.

Contribution

The paper proposes WorldRFT, a novel framework that aligns scene representation with planning using hierarchical decomposition, local refinement, and reinforcement learning fine-tuning for autonomous driving.

Findings

Achieves state-of-the-art results on nuScenes and NavSim benchmarks.

Reduces collision rates by 83% on nuScenes.

Performs competitively with LiDAR-based methods using camera-only input.

Abstract

Latent World Models enhance scene representation through temporal self-supervised learning, presenting a perception annotation-free paradigm for end-to-end autonomous driving. However, the reconstruction-oriented representation learning tangles perception with planning tasks, leading to suboptimal optimization for planning. To address this challenge, we propose WorldRFT, a planning-oriented latent world model framework that aligns scene representation learning with planning via a hierarchical planning decomposition and local-aware interactive refinement mechanism, augmented by reinforcement learning fine-tuning (RFT) to enhance safety-critical policy performance. Specifically, WorldRFT integrates a vision-geometry foundation model to improve 3D spatial awareness, employs hierarchical planning task decomposition to guide representation optimization, and utilizes local-aware iterative refinement to derive a planning-oriented driving policy. Furthermore, we introduce Group Relative Policy Optimization (GRPO), which applies trajectory Gaussianization and collision-aware rewards to fine-tune the driving policy, yielding systematic improvements in safety. WorldRFT achieves state-of-the-art (SOTA) performance on both open-loop nuScenes and closed-loop NavSim benchmarks. On nuScenes, it reduces collision rates by 83% (0.30% -> 0.05%). On NavSim, using camera-only sensors input, it attains competitive performance with the LiDAR-based SOTA method DiffusionDrive (87.8 vs. 88.1 PDMS).