A Data-driven Pricing Scheme for Optimal Routing through Artificial Currencies

TL;DR

This paper introduces a data-driven, reinforcement learning-based artificial currency pricing scheme that aligns selfish user behavior with system-optimal routing, reducing societal costs in mobility systems.

Contribution

It presents a novel adaptive artificial currency tolling method using reinforcement learning to achieve system optimality in routing with fairness considerations.

Findings

Reduces societal costs by up to 25% compared to uncontrolled flows.

Effectively aligns user flows with the system optimum in various scenarios.

Responds robustly to environmental changes.

Abstract

Mobility systems often suffer from a high price of anarchy due to the uncontrolled behavior of selfish users. This may result in societal costs that are significantly higher compared to what could be achieved by a centralized system-optimal controller. Monetary tolling schemes can effectively align the behavior of selfish users with the system-optimum. Yet, they inevitably discriminate the population in terms of income. Artificial currencies were recently presented as an effective alternative that can achieve the same performance, whilst guaranteeing fairness among the population. However, those studies were based on behavioral models that may differ from practical implementations. This paper presents a data-driven approach to automatically adapt artificial-currency tolls within repetitive-game settings. We first consider a parallel-arc setting whereby users commute on a daily basis from an individual origin to an individual destination, choosing a route in exchange of an artificial-currency price or reward, while accounting for the impact of the choices of the other users on travel discomfort. Second, we devise a model-based reinforcement learning controller that autonomously learns the optimal pricing policy by interacting with the proposed framework considering the closeness of the observed aggregate flows to a desired system-optimal distribution as a reward function. Our numerical results show that the proposed data-driven pricing scheme can effectively align the users' flows with the system optimum, significantly reducing the societal costs with respect to the uncontrolled flows (by about 15% and 25% depending on the scenario), and respond to environmental changes in a robust and efficient manner.