Auto-Encoding Goodness of Fit

TL;DR

The paper introduces the Goodness-of-Fit Autoencoder (GoFAE), a novel generative model that integrates GoF tests at multiple levels to improve latent space regularization and generative quality.

Contribution

It proposes a new autoencoder framework that uses GoF tests for regularization and parameter selection, with theoretical guarantees and empirical validation.

Findings

Achieves competitive FID scores and MSE with existing models.

Balances reconstruction and generation via mutual information and p-value uniformity.

Maintains statistical indistinguishability from Gaussian in latent space.

Abstract

We develop a new type of generative autoencoder called the Goodness-of-Fit Autoencoder (GoFAE), which incorporates GoF tests at two levels. At the minibatch level, it uses GoF test statistics as regularization objectives. At a more global level, it selects a regularization coefficient based on higher criticism, i.e., a test on the uniformity of the local GoF p-values. We justify the use of GoF tests by providing a relaxed $L_2$-Wasserstein bound on the distance between the latent distribution and a distribution class. We prove that optimization based on these tests can be done with stochastic gradient descent on a compact Riemannian manifold. Empirically, we show that our higher criticism parameter selection procedure balances reconstruction and generation using mutual information and uniformity of p-values respectively. Finally, we show that GoFAE achieves comparable FID scores and mean squared errors with competing deep generative models while retaining statistical indistinguishability from Gaussian in the latent space based on a variety of hypothesis tests.