Perturbative estimation of stochastic gradients

TL;DR

This paper introduces perturbative stochastic gradient estimators that improve convergence in variational inference, applicable to both continuous and discrete functions, with variance reduction techniques and extensions to binary-weighted networks.

Contribution

The paper presents a novel perturbative expansion approach for stochastic gradient estimation, including bias and variance analysis, variance reduction methods, and extensions to discrete functions via analytic continuation.

Findings

Perturbative estimators reduce bias and variance in stochastic gradients.

The methods improve convergence in stochastic variational inference.

Extensions enable training of stochastic networks with binary weights.

Abstract

In this paper we introduce a family of stochastic gradient estimation techniques based of the perturbative expansion around the mean of the sampling distribution. We characterize the bias and variance of the resulting Taylor-corrected estimators using the Lagrange error formula. Furthermore, we introduce a family of variance reduction techniques that can be applied to other gradient estimators. Finally, we show that these new perturbative methods can be extended to discrete functions using analytic continuation. Using this technique, we derive a new gradient descent method for training stochastic networks with binary weights. In our experiments, we show that the perturbative correction improves the convergence of stochastic variational inference both in the continuous and in the discrete case.