Cooperative Guidance Strategy for Active Defense Spacecraft with Imperfect Information via Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning-based cooperative guidance strategy for active spacecraft defense, demonstrating improved adaptability and performance over traditional optimal guidance methods in simulated multi-agent scenarios.

Contribution

The paper develops a novel deep reinforcement learning approach with a universal reward design and training method for active spacecraft defense, outperforming classical strategies.

Findings

Enhanced target escape and win rates in simulations.

Superior adaptability to interceptor maneuverability.

Faster convergence and reduced overfitting in training.

Abstract

In this paper, an adaptive cooperative guidance strategy for the active protection of a target spacecraft trying to evade an interceptor was developed. The target spacecraft performs evasive maneuvers, launching an active defense vehicle to divert the interceptor. Instead of classical strategies, which are based on optimal control or differential game theory, the problem was solved by using the deep reinforcement learning method, and imperfect information was assumed for the interceptor maneuverability. To address the sparse reward problem, a universal reward design method and an increasingly difficult training approach were presented utilizing the shaping technique. Guidance law, reward function, and training approach were demonstrated through the learning process and Monte Carlo simulations. The application of the non-sparse reward function and increasingly difficult training approach accelerated the model convergence, alleviating the overfitting problem. Considering a standard optimal guidance law as a benchmark, the effectiveness, and the advantages, that guarantee the target spacecraft's escape and win rates in a multi-agent game, of the proposed guidance strategy were validated by the simulation results. The trained agent's adaptiveness to the interceptor maneuverability was superior to the optimal guidance law. Moreover, compared to the standard optimal guidance law, the proposed guidance strategy performed better with less prior knowledge.