Crossing the Sim2Real Gap Between Simulation and Ground Testing to Space Deployment of Autonomous Free-flyer Control

TL;DR

This paper demonstrates the first on-orbit deployment of reinforcement learning for autonomous control of a space robot, successfully bridging the simulation-to-reality gap using NVIDIA's Omniverse simulator.

Contribution

It introduces a novel training pipeline that enables efficient RL policy transfer from simulation to space environment for the Astrobee robot.

Findings

RL-based control was successfully deployed on the ISS.

The training pipeline effectively bridged the Sim2Real gap.

The approach enables rapid on-orbit adaptation for space robots.

Abstract

Reinforcement learning (RL) offers transformative potential for robotic control in space. We present the first on-orbit demonstration of RL-based autonomous control of a free-flying robot, the NASA Astrobee, aboard the International Space Station (ISS). Using NVIDIA's Omniverse physics simulator and curriculum learning, we trained a deep neural network to replace Astrobee's standard attitude and translation control, enabling it to navigate in microgravity. Our results validate a novel training pipeline that bridges the simulation-to-reality (Sim2Real) gap, utilizing a GPU-accelerated, scientific-grade simulation environment for efficient Monte Carlo RL training. This successful deployment demonstrates the feasibility of training RL policies terrestrially and transferring them to space-based applications. This paves the way for future work in In-Space Servicing, Assembly, and Manufacturing (ISAM), enabling rapid on-orbit adaptation to dynamic mission requirements.