Stereo-GS: Multi-View Stereo Vision Model for Generalizable 3D Gaussian Splatting Reconstruction

TL;DR

This paper introduces Stereo-GS, a novel framework for efficient, pose-free 3D Gaussian Splatting reconstruction that disentangles geometry and appearance prediction, reducing computational demands and improving robustness.

Contribution

The paper presents a disentangled, resource-efficient 3D Gaussian prediction method using stereo vision and attention mechanisms, enabling pose-free reconstruction without relying on camera parameters.

Findings

Achieves high-quality 3D reconstruction with less computational resources.

Operates effectively without camera pose information.

Provides scalable and robust 3D content generation.

Abstract

Generalizable 3D Gaussian Splatting reconstruction showcases advanced Image-to-3D content creation but requires substantial computational resources and large datasets, posing challenges to training models from scratch. Current methods usually entangle the prediction of 3D Gaussian geometry and appearance, which rely heavily on data-driven priors and result in slow regression speeds. To address this, we propose \method, a disentangled framework for efficient 3D Gaussian prediction. Our method extracts features from local image pairs using a stereo vision backbone and fuses them via global attention blocks. Dedicated point and Gaussian prediction heads generate multi-view point-maps for geometry and Gaussian features for appearance, combined as GS-maps to represent the 3DGS object. A refinement network enhances these GS-maps for high-quality reconstruction. Unlike existing methods that depend on camera parameters, our approach achieves pose-free 3D reconstruction, improving robustness and practicality. By reducing resource demands while maintaining high-quality outputs, \method provides an efficient, scalable solution for real-world 3D content generation.