Importance Sampling in Expensive Finite-Sum Optimization via Contextual Bandit Methods

TL;DR

This paper explores using contextual bandit algorithms, specifically Exp4, to optimize sampling strategies in expensive finite-sum problems with multiple simulation outputs, leveraging side information.

Contribution

It introduces a novel approach to generate sampling distributions for SAM methods by framing the problem as a contextual bandit task, incorporating side information.

Findings

Preliminary numerical results on synthetic problems demonstrate potential benefits.

Using Exp4 can improve subset selection in SAM methods.

Side information can be effectively integrated into the sampling process.

Abstract

In computational science workflows, it is often the case that 1) objective functions for optimization involve multiple simulation outputs, and 2) those simulations can be performed (at least partially) in parallel. In this work, we reexamine past work on a class of randomized algorithms, stochastic average model (SAM) methods. SAM methods are conceptually similar to stochastic average gradient methods, and effectively require that only randomized subsets of simulation outputs be locally modeled in each iteration of a model-based optimization method. This work focuses on the question of how best to perform this randomization of subset selection, especially in settings where there exists useful side information such as alternative lower-fidelity simulations, pre-trained emulators or domain expertise from humans or AI models. In particular, we consider the problem of generating sampling distributions for SAM methods as a contextual bandit problem and we apply the Exponential weights algorithm for Exploration and Exploitation with Experts (Exp4). We provide some preliminary numerical results on synthetic problems.