Deep Reinforcement Learning for Scalable Multiagent Spacecraft Inspection

TL;DR

This paper introduces a scalable, lidar-like observation space for multiagent reinforcement learning, enabling autonomous spacecraft teams to efficiently inspect satellites without increasing observation complexity as team size varies.

Contribution

It proposes a novel scalable observation method for multiagent RL that maintains constant input size regardless of the number of agents, improving coordination in spacecraft inspection tasks.

Findings

Scalable observation space improves multiagent cooperation.

RL agents successfully perform spacecraft inspection tasks.

Communication enhances efficiency in multi-spacecraft missions.

Abstract

As the number of spacecraft in orbit continues to increase, it is becoming more challenging for human operators to manage each mission. As a result, autonomous control methods are needed to reduce this burden on operators. One method of autonomous control is Reinforcement Learning (RL), which has proven to have great success across a variety of complex tasks. For missions with multiple controlled spacecraft, or agents, it is critical for the agents to communicate and have knowledge of each other, where this information is typically given to the Neural Network Controller (NNC) as an input observation. As the number of spacecraft used for the mission increases or decreases, rather than modifying the size of the observation, this paper develops a scalable observation space that uses a constant observation size to give information on all of the other agents. This approach is similar to a lidar sensor, where determines ranges of other objects in the environment. This observation space is applied to a spacecraft inspection task, where RL is used to train multiple deputy spacecraft to cooperate and inspect a passive chief spacecraft. It is expected that the scalable observation space will allow the agents to learn to complete the task more efficiently compared to a baseline solution where no information is communicated between agents.