CAD-Free Learning of Spacecraft Pose Estimators via NeRF-Based Augmentations

TL;DR

This paper presents a NeRF-based augmentation technique that enables spacecraft pose estimation models to be trained effectively with limited real images, eliminating the need for CAD models and improving real-world robustness.

Contribution

The proposed method allows training accurate spacecraft pose estimators from few images using NeRF-generated augmentations, bypassing CAD models and large synthetic datasets.

Findings

Effective pose estimation with only 25-400 real images.

Enhanced robustness to real on-orbit conditions.

Improved out-of-domain generalization with NeRF augmentation.

Abstract

Spacecraft pose estimation networks require tens of thousands of CAD-rendered images to be trained. This reliance on synthetic CAD data (i) limits applicability to targets with reliable geometry prior, excluding uncooperative or poorly documented spacecraft, and (ii) causes poor generalization to real on-orbit conditions due to unrealistic illumination and material appearance. This paper introduces a NeRF-based image augmentation method that enables the learning of spacecraft pose estimators from only a few tens to a few hundreds of images. The method learns a Neural Radiance Field of the target and generates a large, diverse dataset through geometrically-consistent viewpoint and appearance augmentation. This augmented dataset enables the training of accurate target-specific pose estimators without requiring a CAD model or large synthetic datasets. Experiments show that our approach supports the training of accurate pose estimators from only 25 to 400 realistic images, even under severe illumination variations. When applied on large CAD-based synthetic datasets, the NeRF-based augmentation also enhances out-of-domain generalization, yielding improved robustness to real on-orbit conditions.