LeAD: The LLM Enhanced Planning System Converged with End-to-end Autonomous Driving

TL;DR

LeAD is an autonomous driving system that combines high-frequency perception and planning with LLM-based reasoning to better handle complex urban scenarios, improving decision-making and safety.

Contribution

This work introduces a dual-rate architecture integrating LLM augmentation with end-to-end driving, enhancing scenario understanding and decision quality in autonomous vehicles.

Findings

Achieved 71 points on Leaderboard V1 benchmark.

Route completion rate of 93%.

Superior handling of unconventional scenarios in CARLA.

Abstract

A principal barrier to large-scale deployment of urban autonomous driving systems lies in the prevalence of complex scenarios and edge cases. Existing systems fail to effectively interpret semantic information within traffic contexts and discern intentions of other participants, consequently generating decisions misaligned with skilled drivers' reasoning patterns. We present LeAD, a dual-rate autonomous driving architecture integrating imitation learning-based end-to-end (E2E) frameworks with large language model (LLM) augmentation. The high-frequency E2E subsystem maintains real-time perception-planning-control cycles, while the low-frequency LLM module enhances scenario comprehension through multi-modal perception fusion with HD maps and derives optimal decisions via chain-of-thought (CoT) reasoning when baseline planners encounter capability limitations. Our experimental evaluation in the CARLA Simulator demonstrates LeAD's superior handling of unconventional scenarios, achieving 71 points on Leaderboard V1 benchmark, with a route completion of 93%.