Spatiotemporal Decision Transformer for Traffic Coordination

TL;DR

This paper introduces MADT, a multi-agent decision transformer that models traffic signal control as a sequence prediction task, leveraging graph attention and temporal transformers to improve coordination and efficiency.

Contribution

MADT is a novel multi-agent sequence modeling approach that extends the Decision Transformer framework with graph attention and temporal encoding for traffic control.

Findings

Achieves 5-6% reduction in average travel time.

Demonstrates superior coordination among intersections.

Performs well on both synthetic and real-world data.

Abstract

Traffic signal control is a critical challenge in urban transportation, requiring coordination among multiple intersections to optimize network-wide traffic flow. While reinforcement learning has shown promise for adaptive signal control, existing methods struggle with multi-agent coordination and sample efficiency. We introduce MADT (Multi-Agent Decision Transformer), a novel approach that reformulates multi-agent traffic signal control as a sequence modeling problem. MADT extends the Decision Transformer paradigm to multi-agent settings by incorporating: (1) a graph attention mechanism for modeling spatial dependencies between intersections, (2) a|temporal transformer encoder for capturing traffic dynamics, and (3) return-to-go conditioning for target performance specification. Our approach enables offline learning from historical traffic data, with architecture design that facilitates potential online fine-tuning. Experiments on synthetic grid networks and real-world traffic scenarios demonstrate that MADT achieves state-of-the-art performance, reducing average travel time by 5-6% compared to the strongest baseline while exhibiting superior coordination among adjacent intersections.