Autonomous Reasoning for Spacecraft Control: A Large Language Model Framework with Group Relative Policy Optimization

TL;DR

This paper introduces a novel framework combining large language models with group relative policy optimization to develop autonomous spacecraft control policies that are both effective and interpretable across various dynamical systems.

Contribution

It presents a two-stage training approach integrating supervised fine-tuning and GRPO, enabling LLMs to generate feasible control policies with human-readable explanations.

Findings

Successfully applied to linear and nonlinear control problems

Generated interpretable control sequences and explanations

Demonstrated feasibility in complex spacecraft attitude control

Abstract

This paper presents a learning-based guidance-and-control approach that couples a reasoning-enabled Large Language Model (LLM) with Group Relative Policy Optimization (GRPO). A two-stage procedure consisting of Supervised Fine-Tuning (SFT) to learn formatting and control primitives, followed by GRPO for interaction-driven policy improvement, trains controllers for each environment. The framework is demonstrated on four control problems spanning a gradient of dynamical complexity, from canonical linear systems through nonlinear oscillatory dynamics to three-dimensional spacecraft attitude control with gyroscopic coupling and thrust constraints. Results demonstrate that an LLM with explicit reasoning, optimized via GRPO, can synthesize feasible stabilizing policies under consistent training settings across both linear and nonlinear systems. The two-stage training methodology enables models to generate control sequences while providing human-readable explanations of their decision-making process. This work establishes a foundation for applying GRPO-based reasoning to autonomous control systems, with potential applications in aerospace and other safety-critical domains.