Towards Natural Language Communication for Cooperative Autonomous Driving via Self-Play

TL;DR

This paper introduces a novel multi-agent learning approach using large language models for natural language communication among autonomous vehicles, enhancing cooperation and coordination in traffic scenarios.

Contribution

It presents a new method, LLM+Debrief, enabling autonomous vehicles to generate human-understandable natural language messages for better traffic cooperation.

Findings

LLM+Debrief outperforms zero-shot LLM in generating meaningful messages

The approach improves vehicle cooperation and traffic flow

Developed a simulation environment for evaluation

Abstract

Past work has demonstrated that autonomous vehicles can drive more safely if they communicate with one another than if they do not. However, their communication has often not been human-understandable. Using natural language as a vehicle-to-vehicle (V2V) communication protocol offers the potential for autonomous vehicles to drive cooperatively not only with each other but also with human drivers. In this work, we propose a suite of traffic tasks in autonomous driving where vehicles in a traffic scenario need to communicate in natural language to facilitate coordination in order to avoid an imminent collision and/or support efficient traffic flow. To this end, this paper introduces a novel method, LLM+Debrief, to learn a message generation and high-level decision-making policy for autonomous vehicles through multi-agent discussion. To evaluate LLM agents for driving, we developed a gym-like simulation environment that contains a range of driving scenarios. Our experimental results demonstrate that LLM+Debrief is more effective at generating meaningful and human-understandable natural language messages to facilitate cooperation and coordination than a zero-shot LLM agent. Our code and demo videos are available at https://talking-vehicles.github.io/.