Safe Deep Reinforcement Learning for Spacecraft Reorientation with Pointing Keep-Out Constraint

TL;DR

This paper develops a safe deep reinforcement learning approach with a control barrier function safety filter for spacecraft reorientation, ensuring attitude constraints are maintained during maneuvers.

Contribution

It introduces a novel state space representation and reward function, combined with a CBF-based safety filter, to improve safety and effectiveness in spacecraft attitude control.

Findings

Simulation results validate the effectiveness of the state space and reward design.

Reward shaping alone cannot guarantee safety during maneuvers.

The CBF-based safety filter ensures constraint compliance during reorientation.

Abstract

This paper implements deep reinforcement learning (DRL) with a safety filter for spacecraft reorientation control with a single pointing keep-out zone. A new state space representation is designed which includes a compact representation of the attitude constraint zone. A reward function is formulated to achieve the control objective while enforcing the attitude constraint. The soft actor-critic (SAC) algorithm is adopted to handle continuous state and action space. A curriculum learning approach is implemented for agent training. To guarantee the compliance of the attitude constraint, a control barrier function (CBF)-based safety filter is implemented for agent deployment. Simulation results demonstrate the effectiveness of the proposed state space presentation and the designed reward function. Monte Carlo simulations underscore that reward shaping alone cannot guarantee the safety during reorientation maneuver. In contrast, with the CBF-based safety filter, the constraint can be guaranteed during maneuvers.