Intelligent Control of Spacecraft Reaction Wheel Attitude Using Deep Reinforcement Learning

TL;DR

This paper presents a novel deep reinforcement learning approach, TD3-HD, for autonomous satellite attitude control that outperforms traditional controllers and existing DRL algorithms, especially under reaction wheel fault conditions.

Contribution

Introduces TD3-HD, a DRL-based control strategy combining TD3, HER, and DWC, to improve fault tolerance and adaptability in satellite attitude control.

Findings

TD3-HD achieves lower attitude error.

Enhanced stability during reaction wheel faults.

Outperforms traditional PD and other DRL algorithms.

Abstract

Reliable satellite attitude control is essential for the success of space missions, particularly as satellites increasingly operate autonomously in dynamic and uncertain environments. Reaction wheels (RWs) play a pivotal role in attitude control, and maintaining control resilience during RW faults is critical to preserving mission objectives and system stability. However, traditional Proportional Derivative (PD) controllers and existing deep reinforcement learning (DRL) algorithms such as TD3, PPO, and A2C often fall short in providing the real time adaptability and fault tolerance required for autonomous satellite operations. This study introduces a DRL-based control strategy designed to improve satellite resilience and adaptability under fault conditions. Specifically, the proposed method integrates Twin Delayed Deep Deterministic Policy Gradient (TD3) with Hindsight Experience Replay (HER) and Dimension Wise Clipping (DWC) referred to as TD3-HD to enhance learning in sparse reward environments and maintain satellite stability during RW failures. The proposed approach is benchmarked against PD control and leading DRL algorithms. Experimental results show that TD3-HD achieves significantly lower attitude error, improved angular velocity regulation, and enhanced stability under fault conditions. These findings underscore the proposed method potential as a powerful, fault tolerant, onboard AI solution for autonomous satellite attitude control.