Diverse Score Distillation

TL;DR

This paper introduces Diverse Score Distillation (DSD), a novel method that enhances diversity in 3D and 2D generation tasks guided by diffusion models, addressing mode collapse in traditional score distillation.

Contribution

The paper proposes a new score formulation inspired by diffusion sampling that promotes diversity and an approximation method for complex scenarios, improving upon prior score distillation techniques.

Findings

DSD significantly increases sample diversity.

DSD maintains high fidelity in generated outputs.

Empirical validation shows superiority over previous methods.

Abstract

Score distillation of 2D diffusion models has proven to be a powerful mechanism to guide 3D optimization, for example enabling text-based 3D generation or single-view reconstruction. A common limitation of existing score distillation formulations, however, is that the outputs of the (mode-seeking) optimization are limited in diversity despite the underlying diffusion model being capable of generating diverse samples. In this work, inspired by the sampling process in denoising diffusion, we propose a score formulation that guides the optimization to follow generation paths defined by random initial seeds, thus ensuring diversity. We then present an approximation to adopt this formulation for scenarios where the optimization may not precisely follow the generation paths (\eg a 3D representation whose renderings evolve in a co-dependent manner). We showcase the applications of our `Diverse Score Distillation' (DSD) formulation across tasks such as 2D optimization, text-based 3D inference, and single-view reconstruction. We also empirically validate DSD against prior score distillation formulations and show that it significantly improves sample diversity while preserving fidelity.