Constructing unbiased gradient estimators with finite variance for conditional stochastic optimization

TL;DR

This paper develops unbiased gradient estimators with finite variance and cost for conditional stochastic optimization problems involving nested expectations, enabling more reliable stochastic gradient descent methods.

Contribution

It introduces a multilevel Monte Carlo gradient estimator that is unbiased with finite variance and cost for nested expectations, extending stochastic optimization theory.

Findings

Multilevel Monte Carlo estimator is unbiased with finite variance.

The proposed method applies to a class of nested expectation problems.

An alternative unbiased estimator is constructed for a special case.

Abstract

We study stochastic gradient descent for solving conditional stochastic optimization problems, in which an objective to be minimized is given by a parametric nested expectation with an outer expectation taken with respect to one random variable and an inner conditional expectation with respect to the other random variable. The gradient of such a parametric nested expectation is again expressed as a nested expectation, which makes it hard for the standard nested Monte Carlo estimator to be unbiased. In this paper, we show under some conditions that a multilevel Monte Carlo gradient estimator is unbiased and has finite variance and finite expected computational cost, so that the standard theory from stochastic optimization for a parametric (non-nested) expectation directly applies. We also discuss a special case for which yet another unbiased gradient estimator with finite variance and cost can be constructed.