Pathwise Gradient Variance Reduction with Control Variates in Variational Inference

TL;DR

This paper reviews existing variance reduction techniques for pathwise gradient estimators in variational inference and introduces a new zero-variance control variate method that requires minimal assumptions about the variational distribution.

Contribution

The paper proposes a novel zero-variance control variate approach for pathwise gradient estimators that is broadly applicable and less assumption-dependent.

Findings

Existing control variates often rely on integrand approximations.

Current methods are limited to simple variational families.

The proposed method reduces variance with minimal assumptions.

Abstract

Variational inference in Bayesian deep learning often involves computing the gradient of an expectation that lacks a closed-form solution. In these cases, pathwise and score-function gradient estimators are the most common approaches. The pathwise estimator is often favoured for its substantially lower variance compared to the score-function estimator, which typically requires variance reduction techniques. However, recent research suggests that even pathwise gradient estimators could benefit from variance reduction. In this work, we review existing control-variates-based variance reduction methods for pathwise gradient estimators to assess their effectiveness. Notably, these methods often rely on integrand approximations and are applicable only to simple variational families. To address this limitation, we propose applying zero-variance control variates to pathwise gradient estimators. This approach offers the advantage of requiring minimal assumptions about the variational distribution, other than being able to sample from it.