The Generalized Reparameterization Gradient

TL;DR

The paper introduces the generalized reparameterization gradient, extending the reparameterization method to a broader class of distributions, enabling low-variance Monte Carlo gradients for variational inference.

Contribution

It proposes a novel generalized reparameterization technique that applies to more distributions, combining reparameterization and score function gradients for improved variational inference.

Findings

Effective with a single sample for low-variance gradients

Applicable to complex probabilistic models

Extends reparameterization to non-Gaussian distributions

Abstract

The reparameterization gradient has become a widely used method to obtain Monte Carlo gradients to optimize the variational objective. However, this technique does not easily apply to commonly used distributions such as beta or gamma without further approximations, and most practical applications of the reparameterization gradient fit Gaussian distributions. In this paper, we introduce the generalized reparameterization gradient, a method that extends the reparameterization gradient to a wider class of variational distributions. Generalized reparameterizations use invertible transformations of the latent variables which lead to transformed distributions that weakly depend on the variational parameters. This results in new Monte Carlo gradients that combine reparameterization gradients and score function gradients. We demonstrate our approach on variational inference for two complex probabilistic models. The generalized reparameterization is effective: even a single sample from the variational distribution is enough to obtain a low-variance gradient.