On the Design Fundamentals of Diffusion Models: A Survey

TL;DR

This survey provides a detailed review of the fundamental design components of diffusion models, highlighting key factors and offering insights for future research in their development and application.

Contribution

It offers a comprehensive, component-level analysis of diffusion model design factors, filling a gap left by previous higher-level reviews.

Findings

Detailed analysis of forward, reverse, and sampling processes

Identification of key design factors for each component

Guidance for future diffusion model research and implementation

Abstract

Diffusion models are learning pattern-learning systems to model and sample from data distributions with three functional components namely the forward process, the reverse process, and the sampling process. The components of diffusion models have gained significant attention with many design factors being considered in common practice. Existing reviews have primarily focused on higher-level solutions, covering less on the design fundamentals of components. This study seeks to address this gap by providing a comprehensive and coherent review of seminal designable factors within each functional component of diffusion models. This provides a finer-grained perspective of diffusion models, benefiting future studies in the analysis of individual components, the design factors for different purposes, and the implementation of diffusion models.