Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference

TL;DR

This paper introduces a simplified, low-variance gradient estimator for variational inference that remains unbiased and becomes more accurate as the approximation improves, with theoretical and empirical validation.

Contribution

It presents a novel gradient estimator that reduces variance by removing a score function term, applicable to complex variational distributions, enhancing inference stability.

Findings

Variance approaches zero near the true posterior

Unbiased estimator with lower variance than standard methods

Effective for complex variational distributions

Abstract

We propose a simple and general variant of the standard reparameterized gradient estimator for the variational evidence lower bound. Specifically, we remove a part of the total derivative with respect to the variational parameters that corresponds to the score function. Removing this term produces an unbiased gradient estimator whose variance approaches zero as the approximate posterior approaches the exact posterior. We analyze the behavior of this gradient estimator theoretically and empirically, and generalize it to more complex variational distributions such as mixtures and importance-weighted posteriors.