Asynchronous Large Language Model Enhanced Planner for Autonomous Driving

TL;DR

This paper presents AsyncDriver, an asynchronous framework that integrates Large Language Models with real-time autonomous driving planners, reducing computational costs while maintaining high performance in complex scenarios.

Contribution

The paper introduces AsyncDriver, a novel asynchronous LLM-enhanced planning framework that decouples inference processes to improve efficiency and effectiveness in autonomous driving.

Findings

Reduces LLM inference costs by decoupling processes.

Maintains high planning performance in challenging scenarios.

Demonstrates effectiveness on nuPlan benchmarks.

Abstract

Despite real-time planners exhibiting remarkable performance in autonomous driving, the growing exploration of Large Language Models (LLMs) has opened avenues for enhancing the interpretability and controllability of motion planning. Nevertheless, LLM-based planners continue to encounter significant challenges, including elevated resource consumption and extended inference times, which pose substantial obstacles to practical deployment. In light of these challenges, we introduce AsyncDriver, a new asynchronous LLM-enhanced closed-loop framework designed to leverage scene-associated instruction features produced by LLM to guide real-time planners in making precise and controllable trajectory predictions. On one hand, our method highlights the prowess of LLMs in comprehending and reasoning with vectorized scene data and a series of routing instructions, demonstrating its effective assistance to real-time planners. On the other hand, the proposed framework decouples the inference processes of the LLM and real-time planners. By capitalizing on the asynchronous nature of their inference frequencies, our approach have successfully reduced the computational cost introduced by LLM, while maintaining comparable performance. Experiments show that our approach achieves superior closed-loop evaluation performance on nuPlan's challenging scenarios.