Multi-Objective Reinforcement Learning for Large-Scale Mixed Traffic Control

TL;DR

This paper introduces a hierarchical multi-objective reinforcement learning framework for large-scale mixed traffic control, improving efficiency, safety, and fairness through explicit risk signals and strategic routing, validated on real-world networks.

Contribution

It presents a novel hierarchical approach combining local RL-based intersection control with network-level routing, incorporating explicit risk signals and fairness penalties.

Findings

Up to 53% reduction in average wait time

Up to 86% reduction in maximum vehicle starvation

Up to 86% reduction in conflict rate

Abstract

Effective mixed traffic control requires balancing efficiency, fairness, and safety. Existing approaches excel at optimizing efficiency and enforcing safety constraints but lack mechanisms to ensure equitable service, resulting in systematic starvation of vehicles on low-demand approaches. We propose a hierarchical framework combining multi-objective reinforcement learning for local intersection control with strategic routing for network-level coordination. Our approach introduces a Conflict Threat Vector that provides agents with explicit risk signals for proactive conflict avoidance, and a queue parity penalty that ensures equitable service across all traffic streams. Extensive experiments on a real-world network across different robot vehicle (RV) penetration rates demonstrate substantial improvements: up to 53% reductions in average wait time, up to 86% reductions in maximum starvation, and up to 86\% reduction in conflict rate compared to baselines, while maintaining fuel efficiency. Our analysis reveals that strategic routing effectiveness scales with RV penetration, becoming increasingly valuable at higher autonomy levels. The results demonstrate that multi-objective optimization through well-curated reward functions paired with strategic RV routing yields significant benefits in fairness and safety metrics critical for equitable mixed-autonomy deployment.