DDIM Redux: Mathematical Foundation and Some Extension

TL;DR

This paper critically reviews the mathematical foundations of gDDIM and EI schemes, providing exact expressions and insights, and introduces a new principal-axis DDIM scheme for improved performance.

Contribution

It offers enhanced mathematical results, including exact formulas for reverse trajectories and covariance, and proposes a novel paDDIM scheme.

Findings

Exact expression for reverse trajectory in probability flow ODE

Exact covariance matrix in gDDIM scheme

Introduction of principal-axis DDIM (paDDIM) scheme

Abstract

This note provides a critical review of the mathematical concepts underlying the generalized diffusion denoising implicit model (gDDIM) and the exponential integrator (EI) scheme. We present enhanced mathematical results, including an exact expression for the reverse trajectory in the probability flow ODE and an exact expression for the covariance matrix in the gDDIM scheme. Furthermore, we offer an improved understanding of the EI scheme's efficiency in terms of the change of variables. The noising process in DDIM is analyzed from the perspective of non-equilibrium statistical physics. Additionally, we propose a new scheme for DDIM, called the principal-axis DDIM (paDDIM).