Adapting Stereo Vision From Objects To 3D Lunar Surface Reconstruction with the StereoLunar Dataset

TL;DR

This paper introduces LunarStereo, a new photorealistic lunar stereo dataset, and adapts a deep learning model for accurate 3D lunar surface reconstruction, addressing challenges like low texture and lighting variations.

Contribution

The paper presents the first lunar stereo dataset and demonstrates how fine-tuning existing models improves 3D reconstruction in lunar conditions.

Findings

Significant improvement over zero-shot baselines.

Effective adaptation of deep models to lunar data.

Robust 3D surface reconstruction demonstrated.

Abstract

Accurate 3D reconstruction of lunar surfaces is essential for space exploration. However, existing stereo vision reconstruction methods struggle in this context due to the Moon's lack of texture, difficult lighting variations, and atypical orbital trajectories. State-of-the-art deep learning models, trained on human-scale datasets, have rarely been tested on planetary imagery and cannot be transferred directly to lunar conditions. To address this issue, we introduce LunarStereo, the first open dataset of photorealistic stereo image pairs of the Moon, simulated using ray tracing based on high-resolution topography and reflectance models. It covers diverse altitudes, lighting conditions, and viewing angles around the lunar South Pole, offering physically grounded supervision for 3D reconstruction tasks. Based on this dataset, we adapt the MASt3R model to the lunar domain through fine-tuning on LunarStereo. We validate our approach through extensive qualitative and quantitative experiments on both synthetic and real lunar data, evaluating 3D surface reconstruction and relative pose estimation. Extensive experiments on synthetic and real lunar data validate the approach, demonstrating significant improvements over zero-shot baselines and paving the way for robust cross-scale generalization in extraterrestrial environments.