Leveraging Neural Radiance Fields for Pose Estimation of an Unknown Space Object during Proximity Operations

TL;DR

This paper introduces a novel approach that uses Neural Radiance Fields to enable pose estimation of unknown space objects from limited images, facilitating autonomous space rendezvous and debris removal.

Contribution

It presents a method to adapt existing pose estimators to unknown targets using in-the-wild NeRFs trained on sparse images, bridging the gap between synthetic and real-world data.

Findings

Successful pose estimation on Hardware-In-the-Loop images

Comparable performance to models trained on synthetic CAD data

Effective use of NeRFs for diverse illumination and viewpoints

Abstract

We address the estimation of the 6D pose of an unknown target spacecraft relative to a monocular camera, a key step towards the autonomous rendezvous and proximity operations required by future Active Debris Removal missions. We present a novel method that enables an "off-the-shelf" spacecraft pose estimator, which is supposed to known the target CAD model, to be applied on an unknown target. Our method relies on an in-the wild NeRF, i.e., a Neural Radiance Field that employs learnable appearance embeddings to represent varying illumination conditions found in natural scenes. We train the NeRF model using a sparse collection of images that depict the target, and in turn generate a large dataset that is diverse both in terms of viewpoint and illumination. This dataset is then used to train the pose estimation network. We validate our method on the Hardware-In-the-Loop images of SPEED+ that emulate lighting conditions close to those encountered on orbit. We demonstrate that our method successfully enables the training of an off-the-shelf spacecraft pose estimation network from a sparse set of images. Furthermore, we show that a network trained using our method performs similarly to a model trained on synthetic images generated using the CAD model of the target.