Correlation Analysis of Generative Models

TL;DR

This paper introduces a unified linear representation for diffusion and flow matching models, analyzes their correlation weaknesses, and discusses implications for model prediction and learning processes.

Contribution

It proposes a unified linear framework for diffusion and flow matching models and provides theoretical insights into their correlation issues.

Findings

Correlation between noisy data and predicted target can be weak in existing models

Weak correlation may impact the effectiveness of prediction and learning

Unified representation simplifies understanding of different generative models

Abstract

Based on literature review about existing diffusion models and flow matching with a neural network to predict a predefined target from noisy data, a unified representation is first proposed for these models using two simple linear equations in this paper. Theoretical analysis of the proposed model is then presented. Our theoretical analysis shows that the correlation between the noisy data and the predicted target is sometimes weak in the existing diffusion models and flow matching. This might affect the prediction (or learning) process which plays a crucial role in all models.