# Dirichlet Variational Autoencoder

## TL;DR

This paper introduces Dirichlet Variational Autoencoder (DirVAE), which uses a Dirichlet prior for continuous latent variables, improving interpretability and avoiding collapsing issues, with superior performance on various tasks.

## Contribution

The paper proposes DirVAE with a Dirichlet prior and a stochastic gradient inference method, addressing component collapsing and enhancing interpretability and performance.

## Key findings

- DirVAE models achieve the best log-likelihood among baselines.
- DirVAE produces more interpretable latent representations.
- DirVAE outperforms baselines in semi-supervised and supervised classification tasks.

## Abstract

This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a Dirichlet prior for a continuous latent variable that exhibits the characteristic of the categorical probabilities. To infer the parameters of DirVAE, we utilize the stochastic gradient method by approximating the Gamma distribution, which is a component of the Dirichlet distribution, with the inverse Gamma CDF approximation. Additionally, we reshape the component collapsing issue by investigating two problem sources, which are decoder weight collapsing and latent value collapsing, and we show that DirVAE has no component collapsing; while Gaussian VAE exhibits the decoder weight collapsing and Stick-Breaking VAE shows the latent value collapsing. The experimental results show that 1) DirVAE models the latent representation result with the best log-likelihood compared to the baselines; and 2) DirVAE produces more interpretable latent values with no collapsing issues which the baseline models suffer from. Also, we show that the learned latent representation from the DirVAE achieves the best classification accuracy in the semi-supervised and the supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models show better performances in most cases.