Deep reinforcement learning for the dynamic vehicle dispatching problem: An event-based approach

TL;DR

This paper introduces an event-based deep reinforcement learning approach for the dynamic vehicle dispatching problem, modeling it as a semi-Markov decision process to improve decision-making efficiency and effectiveness.

Contribution

The paper presents a novel event-based modeling framework and a deep Q-learning method for dynamic vehicle dispatching, reducing complexity and enhancing performance over traditional heuristics.

Findings

Up to 50% reduction in average waiting times.

Improved cancellation rates and total service times.

Effective in realistic NYC scenarios.

Abstract

The dynamic vehicle dispatching problem corresponds to deciding which vehicles to assign to requests that arise stochastically over time and space. It emerges in diverse areas, such as in the assignment of trucks to loads to be transported; in emergency systems; and in ride-hailing services. In this paper, we model the problem as a semi-Markov decision process, which allows us to treat time as continuous. In this setting, decision epochs coincide with discrete events whose time intervals are random. We argue that an event-based approach substantially reduces the combinatorial complexity of the decision space and overcomes other limitations of discrete-time models often proposed in the literature. In order to test our approach, we develop a new discrete-event simulator and use double deep q-learning to train our decision agents. Numerical experiments are carried out in realistic scenarios using data from New York City. We compare the policies obtained through our approach with heuristic policies often used in practice. Results show that our policies exhibit better average waiting times, cancellation rates and total service times, with reduction in average waiting times of up to 50% relative to the other tested heuristic policies.