Statistical Guarantees for Variational Autoencoders using PAC-Bayesian Theory

TL;DR

This paper applies PAC-Bayesian theory to derive statistical guarantees for Variational Autoencoders, including generalization bounds and Wasserstein distance bounds, enhancing understanding of their theoretical properties.

Contribution

It introduces the first PAC-Bayesian bounds for VAEs, providing new theoretical insights into their generalization and distributional approximation capabilities.

Findings

Derived PAC-Bayesian bounds for VAEs' posterior distributions

Established generalization guarantees for reconstruction loss

Provided upper bounds on Wasserstein distance between distributions

Abstract

Since their inception, Variational Autoencoders (VAEs) have become central in machine learning. Despite their widespread use, numerous questions regarding their theoretical properties remain open. Using PAC-Bayesian theory, this work develops statistical guarantees for VAEs. First, we derive the first PAC-Bayesian bound for posterior distributions conditioned on individual samples from the data-generating distribution. Then, we utilize this result to develop generalization guarantees for the VAE's reconstruction loss, as well as upper bounds on the distance between the input and the regenerated distributions. More importantly, we provide upper bounds on the Wasserstein distance between the input distribution and the distribution defined by the VAE's generative model.