A New Distribution on the Simplex with Auto-Encoding Applications

TL;DR

This paper introduces a novel distribution on the simplex, combining Kumaraswamy and stick-breaking processes, with advantageous properties for deep Bayesian auto-encoding tasks, including symmetry, sparsity, and closed-form reparameterization.

Contribution

The paper develops a new simplex distribution with symmetry and closed-form reparameterization, suitable for deep variational Bayesian auto-encoders, and demonstrates its effectiveness in semi-supervised tasks.

Findings

Achieves competitive performance in auto-encoding tasks

Exhibits symmetry similar to Dirichlet under certain conditions

Provides an exact, closed-form reparameterization

Abstract

We construct a new distribution for the simplex using the Kumaraswamy distribution and an ordered stick-breaking process. We explore and develop the theoretical properties of this new distribution and prove that it exhibits symmetry under the same conditions as the well-known Dirichlet. Like the Dirichlet, the new distribution is adept at capturing sparsity but, unlike the Dirichlet, has an exact and closed form reparameterization--making it well suited for deep variational Bayesian modeling. We demonstrate the distribution's utility in a variety of semi-supervised auto-encoding tasks. In all cases, the resulting models achieve competitive performance commensurate with their simplicity, use of explicit probability models, and abstinence from adversarial training.