Improved 3D Gaussian Splatting of Unknown Spacecraft Structure Using Space Environment Illumination Knowledge

TL;DR

This paper introduces a new method for reconstructing the 3D structure of unknown spacecraft using Gaussian Splatting, enhanced by space environment illumination knowledge to handle dynamic lighting conditions during space rendezvous operations.

Contribution

It presents a novel pipeline that incorporates Sun position priors into 3D Gaussian Splatting to improve geometric and photometric accuracy in space imagery with changing illumination.

Findings

Enhanced 3D models adapt to rapid lighting changes in space

Improved photometric accuracy for better pose estimation

Effective handling of shadows and self-occlusion in space scenes

Abstract

This work presents a novel pipeline to recover the 3D structure of an unknown target spacecraft from a sequence of images captured during Rendezvous and Proximity Operations (RPO) in space. The target's geometry and appearance are represented as a 3D Gaussian Splatting (3DGS) model. However, learning 3DGS requires static scenes, an assumption in contrast to dynamic lighting conditions encountered in spaceborne imagery. The trained 3DGS model can also be used for camera pose estimation through photometric optimization. Therefore, in addition to recovering a geometrically accurate 3DGS model, the photometric accuracy of the rendered images is imperative to downstream pose estimation tasks during the RPO process. This work proposes to incorporate the prior knowledge of the Sun's position, estimated and maintained by the servicer spacecraft, into the training pipeline for improved photometric quality of 3DGS rasterization. Experimental studies demonstrate the effectiveness of the proposed solution, as 3DGS models trained on a sequence of images learn to adapt to rapidly changing illumination conditions in space and reflect global shadowing and self-occlusion.