# WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding

## TL;DR

WiSE-ALE introduces a novel variational lower bound for VAEs that emphasizes aggregate dataset statistics, enabling better reconstruction quality and more flexible latent representations.

## Contribution

It proposes an alternative lower bound for VAEs that focuses on aggregate posterior constraints, leading to improved reconstruction and representation flexibility.

## Key findings

- Achieves state-of-the-art reconstruction quality.
- Retains smooth, compact latent representations.
- Offers an efficient learning algorithm, WiSE-ALE.

## Abstract

Variational Auto-encoders (VAEs) have been very successful as methods for forming compressed latent representations of complex, often high-dimensional, data. In this paper, we derive an alternative variational lower bound from the one common in VAEs, which aims to minimize aggregate information loss. Using our lower bound as the objective function for an auto-encoder enables us to place a prior on the bulk statistics, corresponding to an aggregate posterior for the entire dataset, as opposed to a single sample posterior as in the original VAE. This alternative form of prior constraint allows individual posteriors more flexibility to preserve necessary information for good reconstruction quality. We further derive an analytic approximation to our lower bound, leading to an efficient learning algorithm - WiSE-ALE. Through various examples, we demonstrate that WiSE-ALE can reach excellent reconstruction quality in comparison to other state-of-the-art VAE models, while still retaining the ability to learn a smooth, compact representation.