Evolutionary Discovery of Heuristic Policies for Traffic Signal Control

TL;DR

The paper introduces Temporal Policy Evolution for Traffic, a novel framework that uses large language models to evolve specialized heuristic policies for traffic signal control, combining reasoning and environment-specific optimization without training.

Contribution

It presents a new method leveraging LLMs as an evolution engine with structured state abstraction and credit assignment feedback to generate environment-optimized traffic policies.

Findings

Outperforms traditional heuristics in traffic control tasks.

Creates lightweight, environment-specific policies without training.

Demonstrates robustness and efficiency in real-world traffic scenarios.

Abstract

Traffic Signal Control (TSC) involves a challenging trade-off: classic heuristics are efficient but oversimplified, while Deep Reinforcement Learning (DRL) achieves high performance yet suffers from poor generalization and opaque policies. Online Large Language Models (LLMs) provide general reasoning but incur high latency and lack environment-specific optimization. To address these issues, we propose Temporal Policy Evolution for Traffic (\textbf{\method{}}), which uses LLMs as an evolution engine to derive specialized heuristic policies. The framework introduces two key modules: (1) Structured State Abstraction (SSA), converting high-dimensional traffic data into temporal-logical facts for reasoning; and (2) Credit Assignment Feedback (CAF), tracing flawed micro-decisions to poor macro-outcomes for targeted critique. Operating entirely at the prompt level without training, \method{} yields lightweight, robust policies optimized for specific traffic environments, outperforming both heuristics and online LLM actors.