Connecting Consistency Distillation to Score Distillation for Text-to-3D Generation

TL;DR

This paper analyzes score distillation in text-to-3D generation, introduces Guided Consistency Sampling and Brightness-Equalized Generation to improve detail and fidelity, and demonstrates superior results over state-of-the-art methods.

Contribution

It connects consistency distillation theory to score distillation, proposing GCS and BEG to enhance 3D generation quality.

Findings

GCS improves detail and fidelity in 3D assets.

BEG mitigates brightness oversaturation in rendering.

Proposed methods outperform existing state-of-the-art techniques.

Abstract

Although recent advancements in text-to-3D generation have significantly improved generation quality, issues like limited level of detail and low fidelity still persist, which requires further improvement. To understand the essence of those issues, we thoroughly analyze current score distillation methods by connecting theories of consistency distillation to score distillation. Based on the insights acquired through analysis, we propose an optimization framework, Guided Consistency Sampling (GCS), integrated with 3D Gaussian Splatting (3DGS) to alleviate those issues. Additionally, we have observed the persistent oversaturation in the rendered views of generated 3D assets. From experiments, we find that it is caused by unwanted accumulated brightness in 3DGS during optimization. To mitigate this issue, we introduce a Brightness-Equalized Generation (BEG) scheme in 3DGS rendering. Experimental results demonstrate that our approach generates 3D assets with more details and higher fidelity than state-of-the-art methods. The codes are released at https://github.com/LMozart/ECCV2024-GCS-BEG.