Dive3D: Diverse Distillation-based Text-to-3D Generation via Score Implicit Matching

TL;DR

Dive3D introduces a new framework for text-to-3D generation that replaces traditional KL-based loss with Score Implicit Matching, significantly enhancing diversity, fidelity, and alignment in generated 3D assets.

Contribution

It proposes Score Implicit Matching loss and a unified diffusion distillation and reward-guided optimization approach, improving diversity and quality over existing methods.

Findings

Outperforms prior methods in diversity and visual fidelity

Achieves higher scores on text-asset alignment and plausibility

Demonstrates robustness across various prompts and benchmarks

Abstract

Distilling pre-trained 2D diffusion models into 3D assets has driven remarkable advances in text-to-3D synthesis. However, existing methods typically rely on Score Distillation Sampling (SDS) loss, which involves asymmetric KL divergence--a formulation that inherently favors mode-seeking behavior and limits generation diversity. In this paper, we introduce Dive3D, a novel text-to-3D generation framework that replaces KL-based objectives with Score Implicit Matching (SIM) loss, a score-based objective that effectively mitigates mode collapse. Furthermore, Dive3D integrates both diffusion distillation and reward-guided optimization under a unified divergence perspective. Such reformulation, together with SIM loss, yields significantly more diverse 3D outputs while improving text alignment, human preference, and overall visual fidelity. We validate Dive3D across various 2D-to-3D prompts and find that it consistently outperforms prior methods in qualitative assessments, including diversity, photorealism, and aesthetic appeal. We further evaluate its performance on the GPTEval3D benchmark, comparing against nine state-of-the-art baselines. Dive3D also achieves strong results on quantitative metrics, including text-asset alignment, 3D plausibility, text-geometry consistency, texture quality, and geometric detail.