From Images2Mesh: A 3D Surface Reconstruction Pipeline for Non-Cooperative Space Objects

TL;DR

This paper introduces a neural implicit surface reconstruction pipeline for non-cooperative space objects using monocular images, addressing challenges like background variation and illumination changes in real on-orbit footage.

Contribution

It presents a novel pipeline capable of reconstructing 3D surfaces of space debris from real inspection images, including techniques for background removal and photometric correction.

Findings

Segmentation-based background removal is crucial for camera pose estimation.

Photometric correction improves reconstruction quality in varying illumination.

Performance varies in shadowed regions depending on input illumination conditions.

Abstract

On-orbit inspection imagery is crucial as it enables characterization of non-cooperative resident space objects, providing the geometry and structural condition essential for active debris removal and on-orbit servicing mission planning. However, most existing neural implicit surface reconstruction methods have been confined to synthetic or hardware-in-the-loop data with known camera poses and controlled illumination. In this work, we present a pipeline for neural implicit surface reconstruction of non-cooperative space objects from monocular inspection imagery. We demonstrate it on publicly released ISS inspection footage from the STS-119 mission and publicly released on-orbit inspection footage of an H-IIA rocket upper stage. We find that segmentation-based background removal is essential for successful camera pose estimation from real on-orbit footage, where background variation between frames caused direct processing to fail entirely. We further incorporate photometric correction of per-frame exposure variations and analyze its behavior across datasets, finding that performance in shadowed regions varies with the illumination characteristics of the input footage.