A large-scale, physically-based synthetic dataset for satellite pose estimation

TL;DR

This paper presents DLVS3, a high-fidelity synthetic dataset generator for satellite pose estimation, focusing on the Hubble Space Telescope, to improve deep learning models with realistic, diverse training data.

Contribution

The work introduces DLVS3-HST-V1, a novel, physically-based synthetic dataset with detailed annotations, advancing satellite pose estimation research.

Findings

Generated large-scale, realistic datasets with ground-truth pose data

Enabled improved training of deep learning models for satellite pose estimation

Bridged the domain gap between simulation and real-world satellite imagery

Abstract

The Deep Learning Visual Space Simulation System (DLVS3) introduces a novel synthetic dataset generator and a simulation pipeline specifically designed for training and testing satellite pose estimation solutions. This work introduces the DLVS3-HST-V1 dataset, which focuses on the Hubble Space Telescope (HST) as a complex, articulated target. The dataset is generated using advanced real-time and offline rendering technologies, integrating high-fidelity 3D models, dynamic lighting (including secondary sources like Earth reflection), and physically accurate material properties. The pipeline supports the creation of large-scale, richly annotated image sets with ground-truth 6-DoF pose and keypoint data, semantic segmentation, depth, and normal maps. This enables the training and benchmarking of deep learning-based pose estimation solutions under realistic, diverse, and challenging visual conditions. The paper details the dataset generation process, the simulation architecture, and the integration with deep learning frameworks, and positions DLVS3 as a significant step toward closing the domain gap for autonomous spacecraft operations in proximity and servicing missions.