On the Feature Learning in Diffusion Models

TL;DR

This paper investigates the feature learning processes in diffusion models, revealing they develop more balanced data representations than traditional classifiers, supported by theoretical analysis and empirical validation.

Contribution

It introduces a theoretical framework for understanding feature learning in diffusion models and compares it with classification models, highlighting their distinct learning dynamics.

Findings

Diffusion models learn more balanced and comprehensive data representations.

Classification models tend to focus on easy-to-learn data patterns.

Empirical results validate the theoretical differences in feature learning dynamics.

Abstract

The predominant success of diffusion models in generative modeling has spurred significant interest in understanding their theoretical foundations. In this work, we propose a feature learning framework aimed at analyzing and comparing the training dynamics of diffusion models with those of traditional classification models. Our theoretical analysis demonstrates that diffusion models, due to the denoising objective, are encouraged to learn more balanced and comprehensive representations of the data. In contrast, neural networks with a similar architecture trained for classification tend to prioritize learning specific patterns in the data, often focusing on easy-to-learn components. To support these theoretical insights, we conduct several experiments on both synthetic and real-world datasets, which empirically validate our findings and highlight the distinct feature learning dynamics in diffusion models compared to classification.