LucidDreamer: Towards High-Fidelity Text-to-3D Generation via Interval Score Matching

TL;DR

This paper introduces Interval Score Matching (ISM), a novel method that improves text-to-3D generation quality by addressing over-smoothing issues in Score Distillation Sampling, and incorporates 3D Gaussian Splatting for enhanced results.

Contribution

The paper proposes ISM, a new deterministic score matching technique that reduces over-smoothing in text-to-3D models and integrates 3D Gaussian Splatting for superior quality.

Findings

ISM outperforms existing methods in quality.

The approach enhances training efficiency.

Results demonstrate significant improvements in 3D detail and fidelity.

Abstract

The recent advancements in text-to-3D generation mark a significant milestone in generative models, unlocking new possibilities for creating imaginative 3D assets across various real-world scenarios. While recent advancements in text-to-3D generation have shown promise, they often fall short in rendering detailed and high-quality 3D models. This problem is especially prevalent as many methods base themselves on Score Distillation Sampling (SDS). This paper identifies a notable deficiency in SDS, that it brings inconsistent and low-quality updating direction for the 3D model, causing the over-smoothing effect. To address this, we propose a novel approach called Interval Score Matching (ISM). ISM employs deterministic diffusing trajectories and utilizes interval-based score matching to counteract over-smoothing. Furthermore, we incorporate 3D Gaussian Splatting into our text-to-3D generation pipeline. Extensive experiments show that our model largely outperforms the state-of-the-art in quality and training efficiency.