Score-based Denoising Diffusion with Non-Isotropic Gaussian Noise Models

TL;DR

This paper explores denoising diffusion models with non-isotropic Gaussian noise, providing mathematical derivations and initial empirical results that suggest potential improvements over standard isotropic models.

Contribution

It introduces a mathematical framework for non-isotropic Gaussian noise in diffusion models and demonstrates initial empirical viability on CIFAR-10.

Findings

Non-isotropic Gaussian noise can be integrated into diffusion models.

Initial experiments show high-quality samples with non-isotropic noise.

The approach generalizes standard diffusion modeling techniques.

Abstract

Generative models based on denoising diffusion techniques have led to an unprecedented increase in the quality and diversity of imagery that is now possible to create with neural generative models. However, most contemporary state-of-the-art methods are derived from a standard isotropic Gaussian formulation. In this work we examine the situation where non-isotropic Gaussian distributions are used. We present the key mathematical derivations for creating denoising diffusion models using an underlying non-isotropic Gaussian noise model. We also provide initial experiments with the CIFAR-10 dataset to help verify empirically that this more general modeling approach can also yield high-quality samples.