Multi-Agent Reinforcement Learning for Heterogeneous Satellite Cluster Resources Optimization

TL;DR

This paper explores the use of multi-agent reinforcement learning to optimize resource management in a heterogeneous satellite cluster performing autonomous Earth Observation missions, addressing real-time decision-making challenges.

Contribution

It formulates a comprehensive multi-satellite optimization framework and evaluates advanced MARL algorithms in a realistic simulation environment for autonomous satellite coordination.

Findings

MARL enables effective coordination among heterogeneous satellites

It balances imaging performance and resource utilization

MARL improves stability and scalability in satellite operations

Abstract

This work investigates resource optimization in heterogeneous satellite clusters performing autonomous Earth Observation (EO) missions using Reinforcement Learning (RL). In the proposed setting, two optical satellites and one Synthetic Aperture Radar (SAR) satellite operate cooperatively in low Earth orbit to capture ground targets and manage their limited onboard resources efficiently. Traditional optimization methods struggle to handle the real-time, uncertain, and decentralized nature of EO operations, motivating the use of RL and Multi-Agent Reinforcement Learning (MARL) for adaptive decision-making. This study systematically formulates the optimization problem from single-satellite to multi-satellite scenarios, addressing key challenges including energy and memory constraints, partial observability, and agent heterogeneity arising from diverse payload capabilities. Using a near-realistic simulation environment built on the Basilisk and BSK-RL frameworks, we evaluate the performance and stability of state-of-the-art MARL algorithms such as MAPPO, HAPPO, and HATRPO. Results show that MARL enables effective coordination across heterogeneous satellites, balancing imaging performance and resource utilization while mitigating non-stationarity and inter-agent reward coupling. The findings provide practical insights into scalable, autonomous satellite operations and contribute a foundation for future research on intelligent EO mission planning under heterogeneous and dynamic conditions.