Quadrature Compound: An approximating family of distributions

TL;DR

The paper introduces quadrature compounds, a new family of distributions approximating intractable integrals, enabling gradient-based optimization and reparameterization, demonstrated with novel distributions like diffeomixture.

Contribution

It proposes a quadrature-based approximation method for compound distributions that facilitates gradient computation and reparameterization, extending the applicability of such models.

Findings

Quadrature compounds enable efficient gradient estimation.

The method applies to both discrete and continuous distributions.

Introduction of the diffeomixture as a reparameterizable mixture approximation.

Abstract

Compound distributions allow construction of a rich set of distributions. Typically they involve an intractable integral. Here we use a quadrature approximation to that integral to define the quadrature compound family. Special care is taken that this approximation is suitable for computation of gradients with respect to distribution parameters. This technique is applied to discrete (Poisson LogNormal) and continuous distributions. In the continuous case, quadrature compound family naturally makes use of parameterized transformations of unparameterized distributions (a.k.a "reparameterization"), allowing for gradients of expectations to be estimated as the gradient of a sample mean. This is demonstrated in a novel distribution, the diffeomixture, which is is a reparameterizable approximation to a mixture distribution.