GECO: Generative Image-to-3D within a SECOnd

TL;DR

GECO is a fast, high-quality 3D generative method that produces 3D meshes from images within a second by combining a two-stage distillation process to improve efficiency and handle uncertainty.

Contribution

GECO introduces a novel two-stage distillation approach for rapid, high-quality 3D image-to-mesh generation, addressing efficiency and uncertainty issues in existing methods.

Findings

Achieves 3D mesh generation within a second.

Maintains high quality comparable to slower methods.

Demonstrates effectiveness through comprehensive experiments.

Abstract

Recent years have seen significant advancements in 3D generation. While methods like score distillation achieve impressive results, they often require extensive per-scene optimization, which limits their time efficiency. On the other hand, reconstruction-based approaches are more efficient but tend to compromise quality due to their limited ability to handle uncertainty. We introduce GECO, a novel method for high-quality 3D generative modeling that operates within a second. Our approach addresses the prevalent issues of uncertainty and inefficiency in existing methods through a two-stage approach. In the first stage, we train a single-step multi-view generative model with score distillation. Then, a second-stage distillation is applied to address the challenge of view inconsistency in the multi-view generation. This two-stage process ensures a balanced approach to 3D generation, optimizing both quality and efficiency. Our comprehensive experiments demonstrate that GECO achieves high-quality image-to-3D mesh generation with an unprecedented level of efficiency. We will make the code and model publicly available.