NeRF-based Spacecraft Reconstruction from Monocular Imagery Under Illumination Variability and Pose Uncertainty

TL;DR

This paper introduces an extension to Neural Radiance Fields that incorporates learnable appearance embeddings and pose corrections, enabling robust 3D reconstruction of spacecraft from monocular images despite illumination changes and pose uncertainties.

Contribution

The authors propose a novel NeRF extension with per-image parameters to handle illumination variability and pose inaccuracies, improving robustness in spacecraft reconstruction tasks.

Findings

Enhanced NeRF robustness to lighting changes and pose errors

Validated on three in-orbit image datasets

Demonstrated potential for online spacecraft reconstruction

Abstract

Autonomous rendezvous and proximity operations around uncooperative, unknown spacecraft are critical for active debris removal and on-orbit servicing missions. A key component of such operations is the offline reconstruction of a 3D model of the target from a set of 2D images. This task is challenging due to two main factors. First, in-orbit illumination conditions exhibit considerable variability, and change rapidly over time. Second, the inaccuracy of pose information in the images, results in 3D reconstruction uncertainty. To overcome these challenges, we propose to extend Neural Radiance Fields with per-image degrees of freedom: a learnable appearance embedding that captures the illumination conditions specific to each image, and an image-specific pose correction term that refines its noisy pose label to increase 3D consistency across images. These parameters add minimal complexity, as they are learned jointly with the NeRF, yet they substantially improve robustness to illumination variability and pose inaccuracies. We validate our approach on three image sets representative of in-orbit operations, demonstrating its effectiveness for offline reconstruction and highlighting its suitability for online reconstruction, an open problem in the field.