Rapid Distributed Fine-tuning of a Segmentation Model Onboard Satellites

TL;DR

This paper demonstrates the rapid fine-tuning of a lightweight segmentation model onboard satellites using decentralized learning, enabling near-real-time Earth observation analysis during disasters, with promising results in simulated environments.

Contribution

It introduces a method for decentralized fine-tuning of MobileSAM onboard satellites, enhancing rapid response capabilities in Earth observation tasks.

Findings

MobileSAM can be quickly fine-tuned onboard satellites.

Decentralized learning improves segmentation performance.

Fast model updates are feasible with minimal data.

Abstract

Segmentation of Earth observation (EO) satellite data is critical for natural hazard analysis and disaster response. However, processing EO data at ground stations introduces delays due to data transmission bottlenecks and communication windows. Using segmentation models capable of near-real-time data analysis onboard satellites can therefore improve response times. This study presents a proof-of-concept using MobileSAM, a lightweight, pre-trained segmentation model, onboard Unibap iX10-100 satellite hardware. We demonstrate the segmentation of water bodies from Sentinel-2 satellite imagery and integrate MobileSAM with PASEOS, an open-source Python module that simulates satellite operations. This integration allows us to evaluate MobileSAM's performance under simulated conditions of a satellite constellation. Our research investigates the potential of fine-tuning MobileSAM in a decentralised way onboard multiple satellites in rapid response to a disaster. Our findings show that MobileSAM can be rapidly fine-tuned and benefits from decentralised learning, considering the constraints imposed by the simulated orbital environment. We observe improvements in segmentation performance with minimal training data and fast fine-tuning when satellites frequently communicate model updates. This study contributes to the field of onboard AI by emphasising the benefits of decentralised learning and fine-tuning pre-trained models for rapid response scenarios. Our work builds on recent related research at a critical time; as extreme weather events increase in frequency and magnitude, rapid response with onboard data analysis is essential.