Fast model inference and training on-board of Satellites

TL;DR

This paper demonstrates the deployment of a lightweight variational auto-encoder on a CubeSat for real-time image encoding and on-board few-shot training, showcasing fast inference and training capabilities in space.

Contribution

It is the first to deploy a multi-task machine learning model on a CubeSat and perform on-board training, advancing space-based AI applications.

Findings

RaVAEn encodes image tiles in 0.110s onboard.

Successful on-board few-shot training using latent representations.

Deployment on CPU and VPU accelerates processing.

Abstract

Artificial intelligence onboard satellites has the potential to reduce data transmission requirements, enable real-time decision-making and collaboration within constellations. This study deploys a lightweight foundational model called RaVAEn on D-Orbit's ION SCV004 satellite. RaVAEn is a variational auto-encoder (VAE) that generates compressed latent vectors from small image tiles, enabling several downstream tasks. In this work we demonstrate the reliable use of RaVAEn onboard a satellite, achieving an encoding time of 0.110s for tiles of a 4.8x4.8 km$^2$ area. In addition, we showcase fast few-shot training onboard a satellite using the latent representation of data. We compare the deployment of the model on the on-board CPU and on the available Myriad vision processing unit (VPU) accelerator. To our knowledge, this work shows for the first time the deployment of a multi-task model on-board a CubeSat and the on-board training of a machine learning model.