Reinforcement Learning for Mixed Autonomy Intersections

TL;DR

This paper introduces a reinforcement learning approach for managing mixed autonomy traffic at intersections, enabling decentralized control that achieves near-optimal throughput and generalizes across different traffic conditions.

Contribution

It presents a multi-agent reinforcement learning method that coordinates autonomous vehicles at intersections without reward shaping, demonstrating effective traffic management and transferability.

Findings

Achieves 33-50% controlled vehicles with near-optimal throughput.

Vehicles exhibit traffic signal-like behaviors learned through RL.

Behavior generalizes across inflow rates and network sizes.

Abstract

We propose a model-free reinforcement learning method for controlling mixed autonomy traffic in simulated traffic networks with through-traffic-only two-way and four-way intersections. Our method utilizes multi-agent policy decomposition which allows decentralized control based on local observations for an arbitrary number of controlled vehicles. We demonstrate that, even without reward shaping, reinforcement learning learns to coordinate the vehicles to exhibit traffic signal-like behaviors, achieving near-optimal throughput with 33-50% controlled vehicles. With the help of multi-task learning and transfer learning, we show that this behavior generalizes across inflow rates and size of the traffic network. Our code, models, and videos of results are available at https://github.com/ZhongxiaYan/mixed_autonomy_intersections.