Vista3D: Unravel the 3D Darkside of a Single Image

TL;DR

Vista3D is a fast, two-phase framework that generates consistent 3D models from a single image by combining Gaussian Splatting, SDF optimization, and diffusion priors, achieving a balance between diversity and fidelity.

Contribution

The paper introduces Vista3D, a novel method that efficiently produces high-quality 3D reconstructions from a single image using a two-phase approach with disentangled implicit functions.

Findings

Generates 3D objects in under 5 minutes.

Balances diversity and consistency in 3D generation.

Effectively integrates 2D and 3D diffusion priors.

Abstract

We embark on the age-old quest: unveiling the hidden dimensions of objects from mere glimpses of their visible parts. To address this, we present Vista3D, a framework that realizes swift and consistent 3D generation within a mere 5 minutes. At the heart of Vista3D lies a two-phase approach: the coarse phase and the fine phase. In the coarse phase, we rapidly generate initial geometry with Gaussian Splatting from a single image. In the fine phase, we extract a Signed Distance Function (SDF) directly from learned Gaussian Splatting, optimizing it with a differentiable isosurface representation. Furthermore, it elevates the quality of generation by using a disentangled representation with two independent implicit functions to capture both visible and obscured aspects of objects. Additionally, it harmonizes gradients from 2D diffusion prior with 3D-aware diffusion priors by angular diffusion prior composition. Through extensive evaluation, we demonstrate that Vista3D effectively sustains a balance between the consistency and diversity of the generated 3D objects. Demos and code will be available at https://github.com/florinshen/Vista3D.