Robust Multi-Task Learning and Online Refinement for Spacecraft Pose Estimation across Domain Gap

TL;DR

This paper introduces SPNv2, a multi-task CNN for spacecraft pose estimation across different domains, utilizing synthetic training data and online domain refinement to improve accuracy without target domain labels.

Contribution

The work presents a novel multi-task CNN architecture with shared features and an online refinement method that enhances spacecraft pose estimation across domain gaps using synthetic data.

Findings

Joint multi-task training improves feature generalization across domains.

Online Domain Refinement enhances pose estimation accuracy on target images.

Synthetic data training with augmentation reduces the need for real-world labeled data.

Abstract

This work presents Spacecraft Pose Network v2 (SPNv2), a Convolutional Neural Network (CNN) for pose estimation of noncooperative spacecraft across domain gap. SPNv2 is a multi-scale, multi-task CNN which consists of a shared multi-scale feature encoder and multiple prediction heads that perform different tasks on a shared feature output. These tasks are all related to detection and pose estimation of a target spacecraft from an image, such as prediction of pre-defined satellite keypoints, direct pose regression, and binary segmentation of the satellite foreground. It is shown that by jointly training on different yet related tasks with extensive data augmentations on synthetic images only, the shared encoder learns features that are common across image domains that have fundamentally different visual characteristics compared to synthetic images. This work also introduces Online Domain Refinement (ODR) which refines the parameters of the normalization layers of SPNv2 on the target domain images online at deployment. Specifically, ODR performs self-supervised entropy minimization of the predicted satellite foreground, thereby improving the CNN's performance on the target domain images without their pose labels and with minimal computational efforts. The GitHub repository for SPNv2 is available at https://github.com/tpark94/spnv2.