A Multi-Agent LLM Framework for Design Space Exploration in Autonomous Driving Systems

TL;DR

This paper introduces a multi-agent LLM-based framework for automating design space exploration in autonomous driving systems, effectively handling complex configurations and environmental variables to identify optimal solutions.

Contribution

It presents a novel multi-agent LLM framework that automates interpretation and exploration of system designs, improving efficiency over traditional methods.

Findings

Outperforms genetic algorithm baseline in identifying Pareto-optimal solutions.

Reduces navigation time while exploring the design space.

Demonstrates efficiency and effectiveness in a robotaxi case study.

Abstract

Designing autonomous driving systems requires efficient exploration of large hardware/software configuration spaces under diverse environmental conditions, e.g., with varying traffic, weather, and road layouts. Traditional design space exploration (DSE) approaches struggle with multi-modal execution outputs and complex performance trade-offs, and often require human involvement to assess correctness based on execution outputs. This paper presents a multi-agent, large language model (LLM)-based DSE framework, which integrates multi-modal reasoning with 3D simulation and profiling tools to automate the interpretation of execution outputs and guide the exploration of system designs. Specialized LLM agents are leveraged to handle user input interpretation, design point generation, execution orchestration, and analysis of both visual and textual execution outputs, which enables identification of potential bottlenecks without human intervention. A prototype implementation is developed and evaluated on a robotaxi case study (an SAE Level 4 autonomous driving application). Compared with a genetic algorithm baseline, the proposed framework identifies more Pareto-optimal, cost-efficient solutions with reduced navigation time under the same exploration budget. Experimental results also demonstrate the efficiency of the adoption of the LLM-based approach for DSE. We believe that this framework paves the way to the design automation of autonomous driving systems.