Quasi Monte Carlo methods enable extremely low-dimensional deep generative models

TL;DR

This paper presents quasi-Monte Carlo latent variable models (QLVMs) that excel at fitting extremely low-dimensional, interpretable embeddings of high-dimensional data, outperforming traditional VAEs and IWAEs in such settings.

Contribution

Introduction of QLVMs that directly approximate marginal likelihood using quasi-Monte Carlo integration, emphasizing interpretability in low-dimensional latent spaces.

Findings

QLVMs outperform VAEs and IWAEs in low-dimensional datasets.

Embeddings enable visualization, clustering, and density estimation.

Approach is compute-intensive but effective for interpretability.

Abstract

This paper introduces quasi-Monte Carlo latent variable models (QLVMs): a class of deep generative models that are specialized for finding extremely low-dimensional and interpretable embeddings of high-dimensional datasets. Unlike standard approaches, which rely on a learned encoder and variational lower bounds, QLVMs directly approximate the marginal likelihood by randomized quasi-Monte Carlo integration. While this brute force approach has drawbacks in higher-dimensional spaces, we find that it excels in fitting one, two, and three dimensional deep latent variable models. Empirical results on a range of datasets show that QLVMs consistently outperform conventional variational autoencoders (VAEs) and importance weighted autoencoders (IWAEs) with matched latent dimensionality. The resulting embeddings enable transparent visualization and post hoc analyses such as nonparametric density estimation, clustering, and geodesic path computation, which are nontrivial to validate in higher-dimensional spaces. While our approach is compute-intensive and struggles to generate fine-scale details in complex datasets, it offers a compelling solution for applications prioritizing interpretability and latent space analysis.