Environment Optimization for Multi-Agent Navigation

TL;DR

This paper introduces a novel environment optimization framework for multi-agent navigation, using reinforcement learning to improve agent performance while considering environment costs, with proven guarantees and versatile implementation options.

Contribution

It formulates the environment as a decision variable in a system-level optimization, providing a reinforcement learning-based solution with theoretical guarantees and flexible scenarios.

Findings

Environment can be optimized without losing completeness.

Reinforcement learning effectively improves navigation performance.

Numerical results validate the approach's effectiveness.

Abstract

Traditional approaches to the design of multi-agent navigation algorithms consider the environment as a fixed constraint, despite the obvious influence of spatial constraints on agents' performance. Yet hand-designing improved environment layouts and structures is inefficient and potentially expensive. The goal of this paper is to consider the environment as a decision variable in a system-level optimization problem, where both agent performance and environment cost can be accounted for. We begin by proposing a novel environment optimization problem. We show, through formal proofs, under which conditions the environment can change while guaranteeing completeness (i.e., all agents reach their navigation goals). Our solution leverages a model-free reinforcement learning approach. In order to accommodate a broad range of implementation scenarios, we include both online and offline optimization, and both discrete and continuous environment representations. Numerical results corroborate our theoretical findings and validate our approach.