Training Latent Diffusion Models with Interacting Particle Algorithms

TL;DR

This paper presents a new particle-based algorithm for training latent diffusion models, reformulating the process as minimizing a free energy functional and providing theoretical error guarantees.

Contribution

The authors introduce a novel particle interaction algorithm for training latent diffusion models, with theoretical error bounds and improved experimental performance.

Findings

Algorithm compares favorably with previous particle-based methods.

Provides theoretical error guarantees for the training process.

Demonstrates effective end-to-end training of latent diffusion models.

Abstract

We introduce a novel particle-based algorithm for end-to-end training of latent diffusion models. We reformulate the training task as minimizing a free energy functional and obtain a gradient flow that does so. By approximating the latter with a system of interacting particles, we obtain the algorithm, which we underpin theoretically by providing error guarantees. The novel algorithm compares favorably in experiments with previous particle-based methods and variational inference analogues.