Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) International Space Station Astrobee Testing

TL;DR

This paper reports the first successful reinforcement learning control of a free-flying robot in space, demonstrating its potential to enhance autonomous space operations through simulation, ground, and on-orbit testing.

Contribution

It introduces a novel RL-based control policy for space robots, validated through simulation, ground, and actual spaceflight on the ISS, showcasing rapid deployment capabilities.

Findings

RL control policy achieved robust 6-DOF control in space.

Successful on-orbit validation of RL control of a free-flyer.

Demonstrated rapid development and deployment of space robot behaviors.

Abstract

The US Naval Research Laboratory's (NRL's) Autonomous Planning In-space Assembly Reinforcement-learning free-flYer (APIARY) experiment pioneers the use of reinforcement learning (RL) for control of free-flying robots in the zero-gravity (zero-G) environment of space. On Tuesday, May 27th 2025 the APIARY team conducted the first ever, to our knowledge, RL control of a free-flyer in space using the NASA Astrobee robot on-board the International Space Station (ISS). A robust 6-degrees of freedom (DOF) control policy was trained using an actor-critic Proximal Policy Optimization (PPO) network within the NVIDIA Isaac Lab simulation environment, randomizing over goal poses and mass distributions to enhance robustness. This paper details the simulation testing, ground testing, and flight validation of this experiment. This on-orbit demonstration validates the transformative potential of RL for improving robotic autonomy, enabling rapid development and deployment (in minutes to hours) of tailored behaviors for space exploration, logistics, and real-time mission needs.