Controllable Traffic Simulation through LLM-Guided Hierarchical Reasoning and Refinement

TL;DR

This paper introduces a hierarchical, LLM-guided traffic simulation framework that enhances controllability and scenario complexity understanding for autonomous driving evaluation.

Contribution

It presents a diffusion-based, hierarchical reasoning approach with LLMs and a Frenet-frame cost function to improve traffic scenario simulation accuracy and controllability.

Findings

Handles more intricate traffic descriptions

Generates a broader range of scenarios

Improves spatial understanding in simulations

Abstract

Evaluating autonomous driving systems in complex and diverse traffic scenarios through controllable simulation is essential to ensure their safety and reliability. However, existing traffic simulation methods face challenges in their controllability. To address this, we propose a novel diffusion-based and LLM-enhanced traffic simulation framework. Our approach incorporates a high-level understanding module and a low-level refinement module, which systematically examines the hierarchical structure of traffic elements, guides LLMs to thoroughly analyze traffic scenario descriptions step by step, and refines the generation by self-reflection, enhancing their understanding of complex situations. Furthermore, we propose a Frenet-frame-based cost function framework that provides LLMs with geometrically meaningful quantities, improving their grasp of spatial relationships in a scenario and enabling more accurate cost function generation. Experiments on the Waymo Open Motion Dataset (WOMD) demonstrate that our method can handle more intricate descriptions and generate a broader range of scenarios in a controllable manner.