Alignment of Diffusion Models: Fundamentals, Challenges, and Future

TL;DR

This paper reviews the fundamentals, techniques, benchmarks, and challenges in aligning diffusion models with human preferences, highlighting current issues and future research directions in this rapidly evolving field.

Contribution

It is the first comprehensive review that consolidates recent advancements, challenges, and future perspectives on aligning diffusion models with human expectations.

Findings

Summarizes recent alignment techniques for diffusion models.

Identifies key challenges in aligning diffusion models.

Discusses future directions for research in model alignment.

Abstract

Diffusion models have emerged as the leading paradigm in generative modeling, excelling in various applications. Despite their success, these models often misalign with human intentions and generate results with undesired properties or even harmful content. Inspired by the success and popularity of alignment in tuning large language models, recent studies have investigated aligning diffusion models with human expectations and preferences. This work mainly reviews alignment of diffusion models, covering advancements in fundamentals of alignment, alignment techniques of diffusion models, preference benchmarks, and evaluation for diffusion models. Moreover, we discuss key perspectives on current challenges and promising future directions on solving the remaining challenges in alignment of diffusion models. To the best of our knowledge, our work is the first comprehensive review paper for researchers and engineers to comprehend, practice, and research alignment of diffusion models.