Fine-Tuned Language Models as Space Systems Controllers

TL;DR

This paper demonstrates that small, fine-tuned language models can effectively control simplified space systems, requiring less data than traditional neural networks and generalizing well across different control problems.

Contribution

It introduces the novel application of fine-tuned LLMs as controllers for space systems, showing their effectiveness and data efficiency across multiple control tasks.

Findings

Fine-tuned LLMs can control space systems with high precision.

Fine-tuning requires less data than traditional neural networks.

Models generalize well across different control problems.

Abstract

Large language models (LLMs), or foundation models (FMs), are pretrained transformers that coherently complete sentences auto-regressively. In this paper, we show that LLMs can control simplified space systems after some additional training, called fine-tuning. We look at relatively small language models, ranging between 7 and 13 billion parameters. We focus on four problems: a three-dimensional spring toy problem, low-thrust orbit transfer, low-thrust cislunar control, and powered descent guidance. The fine-tuned LLMs are capable of controlling systems by generating sufficiently accurate outputs that are multi-dimensional vectors with up to 10 significant digits. We show that for several problems the amount of data required to perform fine-tuning is smaller than what is generally required of traditional deep neural networks (DNNs), and that fine-tuned LLMs are good at generalizing outside of the training dataset. Further, the same LLM can be fine-tuned with data from different problems, with only minor performance degradation with respect to LLMs trained for a single application. This work is intended as a first step towards the development of a general space systems controller.