Learning Generalizable Multi-Lane Mixed-Autonomy Behaviors in Single Lane Representations of Traffic

TL;DR

This paper develops a curriculum learning approach for traffic control in simplified single-lane models, demonstrating that incorporating lane-changing behaviors enhances the transferability of learned policies to more complex multi-lane traffic systems.

Contribution

It introduces a curriculum learning framework for generalizable traffic control strategies and shows that modeling lane changes improves policy transferability.

Findings

Curriculum learning effectively reduces congestion in simple traffic models.

Including lane change behaviors enhances policy transfer to multi-lane traffic.

Simplified models can inform real-world autonomous driving strategies.

Abstract

Reinforcement learning techniques can provide substantial insights into the desired behaviors of future autonomous driving systems. By optimizing for societal metrics of traffic such as increased throughput and reduced energy consumption, such methods can derive maneuvers that, if adopted by even a small portion of vehicles, may significantly improve the state of traffic for all vehicles involved. These methods, however, are hindered in practice by the difficulty of designing efficient and accurate models of traffic, as well as the challenges associated with optimizing for the behaviors of dozens of interacting agents. In response to these challenges, this paper tackles the problem of learning generalizable traffic control strategies in simple representations of vehicle driving dynamics. In particular, we look to mixed-autonomy ring roads as depictions of instabilities that result in the formation of congestion. Within this problem, we design a curriculum learning paradigm that exploits the natural extendability of the network to effectively learn behaviors that reduce congestion over long horizons. Next, we study the implications of modeling lane changing on the transferability of policies. Our findings suggest that introducing lane change behaviors that even approximately match trends in more complex systems can significantly improve the generalizability of subsequent learned models to more accurate multi-lane models of traffic.