Multi-Agent Environments for Vehicle Routing Problems

TL;DR

This paper introduces MAEnvs4VRP, an open-source, modular framework built on PyTorch for multi-agent vehicle routing problems, facilitating research and comparison in reinforcement learning approaches.

Contribution

It provides a unified, flexible environment supporting various VRP variants, promoting collaboration and innovation between RL and OR communities.

Findings

The framework supports classical, dynamic, stochastic, and multi-task VRP variants.

It features an intuitive API for rapid adoption and integration.

Open-source code is available at https://github.com/ricgama/maenvs4vrp.

Abstract

Research on Reinforcement Learning (RL) approaches for discrete optimization problems has increased considerably, extending RL to areas classically dominated by Operations Research (OR). Vehicle routing problems are a good example of discrete optimization problems with high practical relevance, for which RL techniques have achieved notable success. Despite these advances, open-source development frameworks remain scarce, hindering both algorithm testing and objective comparison of results. This situation ultimately slows down progress in the field and limits the exchange of ideas between the RL and OR communities. Here, we propose MAEnvs4VRP library, a unified framework for multi-agent vehicle routing environments that supports classical, dynamic, stochastic, and multi-task problem variants within a single modular design. The library, built on PyTorch, provides a flexible and modular architecture design that facilitates customization and the incorporation of new routing problems. It follows the Agent Environment Cycle ("AEC") games model and features an intuitive API, enabling rapid adoption and seamless integration into existing reinforcement learning frameworks. The project source code can be found at https://github.com/ricgama/maenvs4vrp.