DreamDPO: Aligning Text-to-3D Generation with Human Preferences via Direct Preference Optimization

TL;DR

DreamDPO introduces a novel optimization framework that aligns text-to-3D content with human preferences using pairwise comparisons and preference-driven loss, resulting in more controllable and higher-quality 3D generation.

Contribution

It presents the first preference optimization approach for text-to-3D generation, improving alignment with human preferences and controllability over existing methods.

Findings

Achieves higher-quality 3D content than existing methods

Enables fine-grained control through preference-guided optimization

Reduces reliance on pointwise quality evaluations

Abstract

Text-to-3D generation automates 3D content creation from textual descriptions, which offers transformative potential across various fields. However, existing methods often struggle to align generated content with human preferences, limiting their applicability and flexibility. To address these limitations, in this paper, we propose DreamDPO, an optimization-based framework that integrates human preferences into the 3D generation process, through direct preference optimization. Practically, DreamDPO first constructs pairwise examples, then compare their alignment with human preferences using reward or large multimodal models, and lastly optimizes the 3D representation with a preference-driven loss function. By leveraging pairwise comparison to reflect preferences, DreamDPO reduces reliance on precise pointwise quality evaluations while enabling fine-grained controllability through preference-guided optimization. Experiments demonstrate that DreamDPO achieves competitive results, and provides higher-quality and more controllable 3D content compared to existing methods. The code and models will be open-sourced.