Mind the Gap: Geometrically Accurate Generative Reconstruction from Disjoint Views

TL;DR

This paper introduces a new approach for 3D reconstruction from disjoint views, leveraging foundation models to synthesize intermediate perspectives and improve geometric accuracy in scenarios lacking visual overlap.

Contribution

It proposes GLADOS, a modular framework for generative reconstruction from disjoint views, and establishes benchmarks and metrics for zero-overlap 3D reconstruction tasks.

Findings

Existing methods fail on disjoint view reconstruction, producing incoherent geometries.

GLADOS effectively synthesizes intermediate views to connect disjoint inputs.

The framework improves geometric coherence in zero-overlap scenarios.

Abstract

3D vision systems are fundamentally constrained by their reliance on visual overlap: reconstruction methods require it for geometric alignment, while generative models use it to enforce multi-view consistency. This limitation is particularly acute in real-world scenarios such as distributed swarm robotics or crowd-sourced data collection, where capturing overlapping perspectives, both in terms of spatial and appearance overlap, is often impossible. We introduce Generative Reconstruction from Disjoint Views as a new paradigm, establish a comprehensive dataset, and propose specialized evaluation metrics for zero-overlap scenarios. Our benchmarking demonstrates that existing state-of-the-art methods fail catastrophically on this task, producing disconnected geometries or semantically incoherent reconstructions. To address these limitations, we propose GLADOS, a general, modular framework that operates through three stages: (1) Generative Bridging, where foundation models synthesize intermediate perspectives to connect disjoint inputs; (2) Robust Coarse 3D Reconstruction, that establish coarse geometric scaffold via global alignment which absorbs local contradictions from generative process; and (3) Iterative Context Expansion and Consistency Optimization to fill missing regions and unify the reconstruction. As an architectureagnostic framework, GLADOS enables seamless integration of future advances in generation, reconstruction, and inpainting. The source code is available at: https://github.com/gwilczynski95/GLADOS.