Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching

TL;DR

This paper introduces a novel method called Optimal Covariance Matching for learning diagonal covariances in probabilistic diffusion models, improving sampling efficiency and likelihood by directly regressing the optimal covariance.

Contribution

It presents a new unbiased objective for directly learning the optimal diagonal covariance in diffusion models, reducing approximation error and enhancing performance.

Findings

Improved sampling efficiency in diffusion models

Enhanced likelihood and recall rate

Significant reduction in covariance approximation error

Abstract

The probabilistic diffusion model has become highly effective across various domains. Typically, sampling from a diffusion model involves using a denoising distribution characterized by a Gaussian with a learned mean and either fixed or learned covariances. In this paper, we leverage the recently proposed covariance moment matching technique and introduce a novel method for learning the diagonal covariance. Unlike traditional data-driven diagonal covariance approximation approaches, our method involves directly regressing the optimal diagonal analytic covariance using a new, unbiased objective named Optimal Covariance Matching (OCM). This approach can significantly reduce the approximation error in covariance prediction. We demonstrate how our method can substantially enhance the sampling efficiency, recall rate and likelihood of commonly used diffusion models.