Synthetic Data for Semantic Image Segmentation of Imagery of Unmanned Spacecraft

TL;DR

This paper introduces a synthetic dataset of spacecraft images for semantic segmentation, demonstrating that models trained on this data can effectively assist autonomous spacecraft operations.

Contribution

The paper presents a novel synthetic data generation method and a dataset for spacecraft image segmentation, enabling research in autonomous space rendezvous.

Findings

Achieved a Sørensen-Dice coefficient of 0.8723 on synthetic data.

Synthetic data enables training effective segmentation models for spacecraft.

Models perform well when target spacecraft and configurations are known.

Abstract

Images of spacecraft photographed from other spacecraft operating in outer space are difficult to come by, especially at a scale typically required for deep learning tasks. Semantic image segmentation, object detection and localization, and pose estimation are well researched areas with powerful results for many applications, and would be very useful in autonomous spacecraft operation and rendezvous. However, recent studies show that these strong results in broad and common domains may generalize poorly even to specific industrial applications on earth. To address this, we propose a method for generating synthetic image data that are labelled for semantic segmentation, generalizable to other tasks, and provide a prototype synthetic image dataset consisting of 2D monocular images of unmanned spacecraft, in order to enable further research in the area of autonomous spacecraft rendezvous. We also present a strong benchmark result (S{\o}rensen-Dice coefficient 0.8723) on these synthetic data, suggesting that it is feasible to train well-performing image segmentation models for this task, especially if the target spacecraft and its configuration are known.