ByPE-VAE: Bayesian Pseudocoresets Exemplar VAE

TL;DR

ByPE-VAE introduces a Bayesian pseudocoreset-based prior to VAE, significantly reducing computational costs while maintaining competitive performance in generative tasks.

Contribution

It proposes a novel Bayesian pseudocoreset prior for VAE, enabling faster training and avoiding overfitting with minimal performance loss.

Findings

Achieves up to 3x faster training than Exemplar VAE.

Maintains comparable density estimation and generative quality.

Effective in representation learning and data augmentation.

Abstract

Recent studies show that advanced priors play a major role in deep generative models. Exemplar VAE, as a variant of VAE with an exemplar-based prior, has achieved impressive results. However, due to the nature of model design, an exemplar-based model usually requires vast amounts of data to participate in training, which leads to huge computational complexity. To address this issue, we propose Bayesian Pseudocoresets Exemplar VAE (ByPE-VAE), a new variant of VAE with a prior based on Bayesian pseudocoreset. The proposed prior is conditioned on a small-scale pseudocoreset rather than the whole dataset for reducing the computational cost and avoiding overfitting. Simultaneously, we obtain the optimal pseudocoreset via a stochastic optimization algorithm during VAE training aiming to minimize the Kullback-Leibler divergence between the prior based on the pseudocoreset and that based on the whole dataset. Experimental results show that ByPE-VAE can achieve competitive improvements over the state-of-the-art VAEs in the tasks of density estimation, representation learning, and generative data augmentation. Particularly, on a basic VAE architecture, ByPE-VAE is up to 3 times faster than Exemplar VAE while almost holding the performance. Code is available at \url{https://github.com/Aiqz/ByPE-VAE}.