Large Language Model guided Deep Reinforcement Learning for Decision Making in Autonomous Driving

TL;DR

This paper introduces a novel LLM-guided deep reinforcement learning framework for autonomous driving decision-making, significantly improving learning efficiency and performance by integrating expert guidance and innovative interaction mechanisms.

Contribution

The study presents a new LLM-guided DRL framework with an expert policy constraint and interaction mechanism, enhancing autonomous driving decision-making efficiency and effectiveness.

Findings

Achieved 90% task success rate in driving scenarios.

Significantly improved learning efficiency over baseline algorithms.

Maintained performance without LLM guidance.

Abstract

Deep reinforcement learning (DRL) shows promising potential for autonomous driving decision-making. However, DRL demands extensive computational resources to achieve a qualified policy in complex driving scenarios due to its low learning efficiency. Moreover, leveraging expert guidance from human to enhance DRL performance incurs prohibitively high labor costs, which limits its practical application. In this study, we propose a novel large language model (LLM) guided deep reinforcement learning (LGDRL) framework for addressing the decision-making problem of autonomous vehicles. Within this framework, an LLM-based driving expert is integrated into the DRL to provide intelligent guidance for the learning process of DRL. Subsequently, in order to efficiently utilize the guidance of the LLM expert to enhance the performance of DRL decision-making policies, the learning and interaction process of DRL is enhanced through an innovative expert policy constrained algorithm and a novel LLM-intervened interaction mechanism. Experimental results demonstrate that our method not only achieves superior driving performance with a 90\% task success rate but also significantly improves the learning efficiency and expert guidance utilization efficiency compared to state-of-the-art baseline algorithms. Moreover, the proposed method enables the DRL agent to maintain consistent and reliable performance in the absence of LLM expert guidance. The code and supplementary videos are available at https://bitmobility.github.io/LGDRL/.