An adaptive importance sampling algorithm for risk-averse optimization

TL;DR

This paper introduces an adaptive importance sampling algorithm that efficiently minimizes CVaR in risk-averse optimization by dynamically adjusting both sample size and distribution, reducing variance and computational cost.

Contribution

The paper presents a novel adaptive sampling method that adjusts sampling distributions on the fly for risk-averse optimization, improving efficiency and convergence.

Findings

Reduces variance of gradient estimates.

Achieves fewer samples per iteration.

Demonstrates computational savings in experiments.

Abstract

Adaptive sampling algorithms are modern and efficient methods that dynamically adjust the sample size throughout the optimization process. However, they may encounter difficulties in risk-averse settings, particularly due to the challenge of accurately sampling from the tails of the underlying distribution of random inputs. This often leads to a much faster growth of the sample size compared to risk-neutral problems. In this work, we propose a novel adaptive sampling algorithm that adapts both the sample size and the sampling distribution at each iteration. The biasing distributions are constructed on the fly, leveraging a reduced-order model of the objective function to be minimized, and are designed to oversample a so-called risk region. As a result, a reduction of the variance of the gradients is achieved, which permits to use fewer samples per iteration compared to a standard algorithm, while still preserving the asymptotic convergence rate. Our focus is on the minimization of the Conditional Value-at-Risk (CVaR), and we establish the convergence of the proposed computational framework. Numerical experiments confirm the substantial computational savings achieved by our approach.