Wonder3D++: Cross-domain Diffusion for High-fidelity 3D Generation from a Single Image

TL;DR

Wonder3D++ is a new cross-domain diffusion approach that efficiently generates high-fidelity textured 3D meshes from a single image by leveraging multi-view normal maps, a multi-view attention mechanism, and a cascaded mesh extraction process.

Contribution

It introduces a cross-domain diffusion model with multi-view normal maps and a cascaded mesh extraction algorithm for improved single-view 3D reconstruction.

Findings

Achieves high-quality 3D reconstructions with robust generalization.

Operates efficiently in about 3 minutes per shape.

Outperforms prior methods in quality and consistency.

Abstract

In this work, we introduce \textbf{Wonder3D++}, a novel method for efficiently generating high-fidelity textured meshes from single-view images. Recent methods based on Score Distillation Sampling (SDS) have shown the potential to recover 3D geometry from 2D diffusion priors, but they typically suffer from time-consuming per-shape optimization and inconsistent geometry. In contrast, certain works directly produce 3D information via fast network inferences, but their results are often of low quality and lack geometric details. To holistically improve the quality, consistency, and efficiency of single-view reconstruction tasks, we propose a cross-domain diffusion model that generates multi-view normal maps and the corresponding color images. To ensure the consistency of generation, we employ a multi-view cross-domain attention mechanism that facilitates information exchange across views and modalities. Lastly, we introduce a cascaded 3D mesh extraction algorithm that drives high-quality surfaces from the multi-view 2D representations in only about $3$ minute in a coarse-to-fine manner. Our extensive evaluations demonstrate that our method achieves high-quality reconstruction results, robust generalization, and good efficiency compared to prior works. Code available at https://github.com/xxlong0/Wonder3D/tree/Wonder3D_Plus.