GeoGS3D: Single-view 3D Reconstruction via Geometric-aware Diffusion Model and Gaussian Splatting

TL;DR

GeoGS3D is a two-stage framework that leverages 2D diffusion models and Gaussian splatting to reconstruct detailed 3D objects from single images, ensuring multi-view consistency and faster processing.

Contribution

It introduces a novel orthogonal plane decomposition and a Gaussian Divergence Significance metric for efficient, high-quality 3D reconstruction from single-view images.

Findings

High multi-view consistency in generated images

Significant acceleration in reconstruction process

Qualitative and quantitative high-quality 3D reconstructions

Abstract

We introduce GeoGS3D, a novel two-stage framework for reconstructing detailed 3D objects from single-view images. Inspired by the success of pre-trained 2D diffusion models, our method incorporates an orthogonal plane decomposition mechanism to extract 3D geometric features from the 2D input, facilitating the generation of multi-view consistent images. During the following Gaussian Splatting, these images are fused with epipolar attention, fully utilizing the geometric correlations across views. Moreover, we propose a novel metric, Gaussian Divergence Significance (GDS), to prune unnecessary operations during optimization, significantly accelerating the reconstruction process. Extensive experiments demonstrate that GeoGS3D generates images with high consistency across views and reconstructs high-quality 3D objects, both qualitatively and quantitatively.