Fast Learning of Non-Cooperative Spacecraft 3D Models through Primitive Initialization

TL;DR

This paper introduces a CNN-based primitive initializer that accelerates the learning of accurate 3D spacecraft models from monocular images, even with noisy poses, reducing training time significantly for space applications.

Contribution

A novel pipeline combining primitive initialization with 3D Gaussian Splatting that reduces training iterations and handles noisy pose estimates in space-related 3D modeling.

Findings

Significantly reduces training iterations and input images needed

Effective with noisy or implicit pose estimates

Enables high-fidelity 3D representations in space applications

Abstract

The advent of novel view synthesis techniques such as NeRF and 3D Gaussian Splatting (3DGS) has enabled learning precise 3D models only from posed monocular images. Although these methods are attractive, they hold two major limitations that prevent their use in space applications: they require poses during training, and have high computational cost at training and inference. To address these limitations, this work contributes: (1) a Convolutional Neural Network (CNN) based primitive initializer for 3DGS using monocular images; (2) a pipeline capable of training with noisy or implicit pose estimates; and (3) and analysis of initialization variants that reduce the training cost of precise 3D models. A CNN takes a single image as input and outputs a coarse 3D model represented as an assembly of primitives, along with the target's pose relative to the camera. This assembly of primitives is then used to initialize 3DGS, significantly reducing the number of training iterations and input images needed -- by at least an order of magnitude. For additional flexibility, the CNN component has multiple variants with different pose estimation techniques. This work performs a comparison between these variants, evaluating their effectiveness for downstream 3DGS training under noisy or implicit pose estimates. The results demonstrate that even with imperfect pose supervision, the pipeline is able to learn high-fidelity 3D representations, opening the door for the use of novel view synthesis in space applications.