H3R: Hybrid Multi-view Correspondence for Generalizable 3D Reconstruction

TL;DR

H3R introduces a hybrid multi-view correspondence framework combining volumetric latent fusion and attention mechanisms, significantly improving generalizable 3D reconstruction speed and accuracy across diverse datasets.

Contribution

The paper proposes a novel hybrid approach that integrates explicit geometric constraints with implicit attention-based refinement, enhancing generalization and convergence speed in 3D reconstruction.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Converges twice as fast as existing methods.

Demonstrates robustness with variable high-resolution views.

Abstract

Despite recent advances in feed-forward 3D Gaussian Splatting, generalizable 3D reconstruction remains challenging, particularly in multi-view correspondence modeling. Existing approaches face a fundamental trade-off: explicit methods achieve geometric precision but struggle with ambiguous regions, while implicit methods provide robustness but suffer from slow convergence. We present H3R, a hybrid framework that addresses this limitation by integrating volumetric latent fusion with attention-based feature aggregation. Our framework consists of two complementary components: an efficient latent volume that enforces geometric consistency through epipolar constraints, and a camera-aware Transformer that leverages Pl\"ucker coordinates for adaptive correspondence refinement. By integrating both paradigms, our approach enhances generalization while converging 2$\times$ faster than existing methods. Furthermore, we show that spatial-aligned foundation models (e.g., SD-VAE) substantially outperform semantic-aligned models (e.g., DINOv2), resolving the mismatch between semantic representations and spatial reconstruction requirements. Our method supports variable-number and high-resolution input views while demonstrating robust cross-dataset generalization. Extensive experiments show that our method achieves state-of-the-art performance across multiple benchmarks, with significant PSNR improvements of 0.59 dB, 1.06 dB, and 0.22 dB on the RealEstate10K, ACID, and DTU datasets, respectively. Code is available at https://github.com/JiaHeng-DLUT/H3R.