End-to-end Deep Reinforcement Learning for Stochastic Multi-objective Optimization in C-VRPTW

TL;DR

This paper introduces an end-to-end deep reinforcement learning model for stochastic multi-objective vehicle routing, effectively handling travel time uncertainty and multiple conflicting goals with improved training efficiency.

Contribution

It proposes a novel deep learning approach that manages stochastic travel times and multiple objectives simultaneously, with a scenario clustering technique to enhance training speed.

Findings

Model constructs high-quality Pareto fronts

Outperforms three baseline methods in run time and solution quality

Effectively handles travel time uncertainty and multiple objectives

Abstract

In this work, we consider learning-based applications in routing to solve a Vehicle Routing variant characterized by stochasticity and multiple objectives. Such problems are representative of practical settings where decision-makers have to deal with uncertainty in the operational environment as well as multiple conflicting objectives due to different stakeholders. We specifically consider travel time uncertainty. We also consider two objectives, total travel time and route makespan, that jointly target operational efficiency and labor regulations on shift length, although different objectives could be incorporated. Learning-based methods offer earnest computational advantages as they can repeatedly solve problems with limited interference from the decision-maker. We specifically focus on end-to-end deep learning models that leverage the attention mechanism and multiple solution trajectories. These models have seen several successful applications in routing problems. However, since travel times are not a direct input to these models due to the large dimensions of the travel time matrix, accounting for uncertainty is a challenge, especially in the presence of multiple objectives. In turn, we propose a model that simultaneously addresses stochasticity and multi-objectivity and provide a refined training mechanism for this model through scenario clustering to reduce training time. Our results show that our model is capable of constructing a Pareto Front of good quality within acceptable run times compared to three baselines.