Multi-Agent Reinforcement Learning for Autonomous Multi-Satellite Earth Observation: A Realistic Case Study

TL;DR

This paper explores the use of Multi-Agent Reinforcement Learning to enable autonomous coordination of satellite constellations for Earth Observation, demonstrating effective resource management and decision-making in a realistic simulation environment.

Contribution

It introduces a MARL framework for multi-satellite EO mission planning, addressing decentralised coordination, partial observability, and resource constraints with empirical evaluation.

Findings

MARL algorithms effectively coordinate multi-satellite tasks

Training stability varies across algorithms like PPO and MAPPO

MARL approaches improve resource management in EO missions

Abstract

The exponential growth of Low Earth Orbit (LEO) satellites has revolutionised Earth Observation (EO) missions, addressing challenges in climate monitoring, disaster management, and more. However, autonomous coordination in multi-satellite systems remains a fundamental challenge. Traditional optimisation approaches struggle to handle the real-time decision-making demands of dynamic EO missions, necessitating the use of Reinforcement Learning (RL) and Multi-Agent Reinforcement Learning (MARL). In this paper, we investigate RL-based autonomous EO mission planning by modelling single-satellite operations and extending to multi-satellite constellations using MARL frameworks. We address key challenges, including energy and data storage limitations, uncertainties in satellite observations, and the complexities of decentralised coordination under partial observability. By leveraging a near-realistic satellite simulation environment, we evaluate the training stability and performance of state-of-the-art MARL algorithms, including PPO, IPPO, MAPPO, and HAPPO. Our results demonstrate that MARL can effectively balance imaging and resource management while addressing non-stationarity and reward interdependency in multi-satellite coordination. The insights gained from this study provide a foundation for autonomous satellite operations, offering practical guidelines for improving policy learning in decentralised EO missions.