Vision-based Neural Scene Representations for Spacecraft

TL;DR

This paper evaluates neural scene representations, specifically NeRF and GRAF, for modeling spacecraft in autonomous space missions, highlighting their strengths in rendering accuracy and pose estimation without prior pose information.

Contribution

It provides a comparative analysis of NeRF and GRAF for spacecraft modeling, demonstrating their respective advantages in rendering and pose estimation in space applications.

Findings

NeRF renders more accurate images of spacecraft material and pose.

GRAF produces detailed novel views without needing pose information.

GRAF effectively handles shadowed spacecraft regions.

Abstract

In advanced mission concepts with high levels of autonomy, spacecraft need to internally model the pose and shape of nearby orbiting objects. Recent works in neural scene representations show promising results for inferring generic three-dimensional scenes from optical images. Neural Radiance Fields (NeRF) have shown success in rendering highly specular surfaces using a large number of images and their pose. More recently, Generative Radiance Fields (GRAF) achieved full volumetric reconstruction of a scene from unposed images only, thanks to the use of an adversarial framework to train a NeRF. In this paper, we compare and evaluate the potential of NeRF and GRAF to render novel views and extract the 3D shape of two different spacecraft, the Soil Moisture and Ocean Salinity satellite of ESA's Living Planet Programme and a generic cube sat. Considering the best performances of both models, we observe that NeRF has the ability to render more accurate images regarding the material specularity of the spacecraft and its pose. For its part, GRAF generates precise novel views with accurate details even when parts of the satellites are shadowed while having the significant advantage of not needing any information about the relative pose.