Sch\"odinger Bridge Type Diffusion Models as an Extension of Variational Autoencoders

TL;DR

This paper introduces a unified framework for Schr"odinger Bridge type diffusion models, interpreting them as an extension of variational autoencoders, which clarifies their objective functions and enhances understanding of their structure.

Contribution

It provides a novel reinterpretation of SB-type diffusion models as an extension of VAEs, simplifying their mathematical understanding and unifying their framework.

Findings

Objective function includes prior loss and drift matching

Reinterprets SB diffusion models within a VAE framework

Clarifies the mathematical structure of SB-type models

Abstract

Generative diffusion models use time-forward and backward stochastic differential equations to connect the data and prior distributions. While conventional diffusion models (e.g., score-based models) only learn the backward process, more flexible frameworks have been proposed to also learn the forward process by employing the Schr\"odinger bridge (SB). However, due to the complexity of the mathematical structure behind SB-type models, we can not easily give an intuitive understanding of their objective function. In this work, we propose a unified framework to construct diffusion models by reinterpreting the SB-type models as an extension of variational autoencoders. In this context, the data processing inequality plays a crucial role. As a result, we find that the objective function consists of the prior loss and drift matching parts.