Dual Policy Reinforcement Learning for Real-time Rebalancing in Bike-sharing Systems

TL;DR

This paper presents a dual policy reinforcement learning approach for real-time bike rebalancing that improves efficiency and demand satisfaction by decoupling inventory and routing decisions, validated through extensive real-world data experiments.

Contribution

It introduces a novel dual policy reinforcement learning framework that separately optimizes inventory and routing decisions for bike-sharing rebalancing, enhancing realism and performance.

Findings

Significant performance improvements over baseline methods.

Effective handling of demand variability influenced by weather and time.

Practical insights for urban mobility operators.

Abstract

Bike-sharing systems play a crucial role in easing traffic congestion and promoting healthier lifestyles. However, ensuring their reliability and user acceptance requires effective strategies for rebalancing bikes. This study introduces a novel approach to address the real-time rebalancing problem with a fleet of vehicles. It employs a dual policy reinforcement learning algorithm that decouples inventory and routing decisions, enhancing realism and efficiency compared to previous methods where both decisions were made simultaneously. We first formulate the inventory and routing subproblems as a multi-agent Markov Decision Process within a continuous time framework. Subsequently, we propose a DQN-based dual policy framework to jointly estimate the value functions, minimizing the lost demand. To facilitate learning, a comprehensive simulator is applied to operate under a first-arrive-first-serve rule, which enables the computation of immediate rewards across diverse demand scenarios. We conduct extensive experiments on various datasets generated from historical real-world data, affected by both temporal and weather factors. Our proposed algorithm demonstrates significant performance improvements over previous baseline methods. It offers valuable practical insights for operators and further explores the incorporation of reinforcement learning into real-world dynamic programming problems, paving the way for more intelligent and robust urban mobility solutions.