Graph Meta-Reinforcement Learning for Transferable Autonomous Mobility-on-Demand

TL;DR

This paper introduces a meta-reinforcement learning approach using graph neural networks to enable autonomous mobility-on-demand systems to quickly adapt to new cities, improving robustness and reducing retraining needs.

Contribution

It formalizes the multi-city AMoD problem with meta-RL and develops an actor-critic algorithm based on recurrent graph neural networks for rapid adaptation.

Findings

Meta-RL policies achieve near-optimal performance in unseen cities.

Control policies adapt rapidly with limited experience in new environments.

Enhanced robustness to real-world distribution shifts.

Abstract

Autonomous Mobility-on-Demand (AMoD) systems represent an attractive alternative to existing transportation paradigms, currently challenged by urbanization and increasing travel needs. By centrally controlling a fleet of self-driving vehicles, these systems provide mobility service to customers and are currently starting to be deployed in a number of cities around the world. Current learning-based approaches for controlling AMoD systems are limited to the single-city scenario, whereby the service operator is allowed to take an unlimited amount of operational decisions within the same transportation system. However, real-world system operators can hardly afford to fully re-train AMoD controllers for every city they operate in, as this could result in a high number of poor-quality decisions during training, making the single-city strategy a potentially impractical solution. To address these limitations, we propose to formalize the multi-city AMoD problem through the lens of meta-reinforcement learning (meta-RL) and devise an actor-critic algorithm based on recurrent graph neural networks. In our approach, AMoD controllers are explicitly trained such that a small amount of experience within a new city will produce good system performance. Empirically, we show how control policies learned through meta-RL are able to achieve near-optimal performance on unseen cities by learning rapidly adaptable policies, thus making them more robust not only to novel environments, but also to distribution shifts common in real-world operations, such as special events, unexpected congestion, and dynamic pricing schemes.