Variational Autoencoders: A Harmonic Perspective

TL;DR

This paper analyzes Variational Autoencoders through harmonic analysis, revealing how encoder variance influences frequency content and robustness, and demonstrating control over these properties via Gaussian noise addition.

Contribution

It introduces a harmonic analysis perspective on VAEs, linking encoder variance to frequency control and robustness, supported by empirical experiments across architectures.

Findings

Larger encoder variances reduce high frequency content.

Increasing variance induces a soft Lipschitz constraint on the decoder.

Adding Gaussian noise allows finer control of frequency and Lipschitz properties.

Abstract

In this work we study Variational Autoencoders (VAEs) from the perspective of harmonic analysis. By viewing a VAE's latent space as a Gaussian Space, a variety of measure space, we derive a series of results that show that the encoder variance of a VAE controls the frequency content of the functions parameterised by the VAE encoder and decoder neural networks. In particular we demonstrate that larger encoder variances reduce the high frequency content of these functions. Our analysis allows us to show that increasing this variance effectively induces a soft Lipschitz constraint on the decoder network of a VAE, which is a core contributor to the adversarial robustness of VAEs. We further demonstrate that adding Gaussian noise to the input of a VAE allows us to more finely control the frequency content and the Lipschitz constant of the VAE encoder networks. To support our theoretical analysis we run experiments with VAEs with small fully-connected neural networks and with larger convolutional networks, demonstrating empirically that our theory holds for a variety of neural network architectures.