Multi Agent Reinforcement Learning for Sequential Satellite Assignment Problems

TL;DR

This paper introduces a multi-agent reinforcement learning approach for sequential assignment problems, particularly in satellite systems, leveraging a known greedy solver to improve decision-making over time.

Contribution

It presents a novel RL-based method that combines known polynomial-time algorithms with learning, enabling scalable and effective solutions for dynamic assignment problems.

Findings

Algorithm outperforms existing methods in realistic scenarios

Scales efficiently to hundreds of agents and tasks

Theoretically justified and avoids common RL pitfalls

Abstract

Assignment problems are a classic combinatorial optimization problem in which a group of agents must be assigned to a group of tasks such that maximum utility is achieved while satisfying assignment constraints. Given the utility of each agent completing each task, polynomial-time algorithms exist to solve a single assignment problem in its simplest form. However, in many modern-day applications such as satellite constellations, power grids, and mobile robot scheduling, assignment problems unfold over time, with the utility for a given assignment depending heavily on the state of the system. We apply multi-agent reinforcement learning to this problem, learning the value of assignments by bootstrapping from a known polynomial-time greedy solver and then learning from further experience. We then choose assignments using a distributed optimal assignment mechanism rather than by selecting them directly. We demonstrate that this algorithm is theoretically justified and avoids pitfalls experienced by other RL algorithms in this setting. Finally, we show that our algorithm significantly outperforms other methods in the literature, even while scaling to realistic scenarios with hundreds of agents and tasks.