Global Rewards in Multi-Agent Deep Reinforcement Learning for Autonomous Mobility on Demand Systems

TL;DR

This paper introduces a novel global-rewards-based multi-agent deep reinforcement learning algorithm for vehicle dispatching in autonomous mobility on demand systems, significantly improving system-wide profit and balancing capabilities over existing local-reward methods.

Contribution

The paper proposes a new global-rewards MADRL algorithm that addresses goal conflicts and enhances performance in AMoD vehicle dispatching tasks.

Findings

Significant performance improvements over state-of-the-art algorithms.

Global rewards enhance vehicle balancing and demand forecasting.

Algorithm tested on real-world data with statistically significant results.

Abstract

We study vehicle dispatching in autonomous mobility on demand (AMoD) systems, where a central operator assigns vehicles to customer requests or rejects these with the aim of maximizing its total profit. Recent approaches use multi-agent deep reinforcement learning (MADRL) to realize scalable yet performant algorithms, but train agents based on local rewards, which distorts the reward signal with respect to the system-wide profit, leading to lower performance. We therefore propose a novel global-rewards-based MADRL algorithm for vehicle dispatching in AMoD systems, which resolves so far existing goal conflicts between the trained agents and the operator by assigning rewards to agents leveraging a counterfactual baseline. Our algorithm shows statistically significant improvements across various settings on real-world data compared to state-of-the-art MADRL algorithms with local rewards. We further provide a structural analysis which shows that the utilization of global rewards can improve implicit vehicle balancing and demand forecasting abilities. Our code is available at https://github.com/tumBAIS/GR-MADRL-AMoD.