Walking the Schr\"odinger Bridge: A Direct Trajectory for Text-to-3D Generation

TL;DR

This paper introduces TraCe, a novel text-to-3D generation method based on Schr"odinger Bridge theory, enabling high-quality 3D assets with less guidance and fewer artifacts by explicitly modeling the optimal transport trajectory.

Contribution

It formulates text-to-3D generation as a Schr"odinger Bridge problem, establishes SDS as a special case, and proposes TraCe for improved, artifact-free 3D generation.

Findings

TraCe outperforms state-of-the-art methods in quality and fidelity.

Theoretical link between SDS and Schr"odinger Bridge is established.

TraCe achieves robust 3D optimization with smaller guidance values.

Abstract

Recent advancements in optimization-based text-to-3D generation heavily rely on distilling knowledge from pre-trained text-to-image diffusion models using techniques like Score Distillation Sampling (SDS), which often introduce artifacts such as over-saturation and over-smoothing into the generated 3D assets. In this paper, we address this essential problem by formulating the generation process as learning an optimal, direct transport trajectory between the distribution of the current rendering and the desired target distribution, thereby enabling high-quality generation with smaller Classifier-free Guidance (CFG) values. At first, we theoretically establish SDS as a simplified instance of the Schr\"odinger Bridge framework. We prove that SDS employs the reverse process of an Schr\"odinger Bridge, which, under specific conditions (e.g., a Gaussian noise as one end), collapses to SDS's score function of the pre-trained diffusion model. Based upon this, we introduce Trajectory-Centric Distillation (TraCe), a novel text-to-3D generation framework, which reformulates the mathematically trackable framework of Schr\"odinger Bridge to explicitly construct a diffusion bridge from the current rendering to its text-conditioned, denoised target, and trains a LoRA-adapted model on this trajectory's score dynamics for robust 3D optimization. Comprehensive experiments demonstrate that TraCe consistently achieves superior quality and fidelity to state-of-the-art techniques.