Adaptive Sampling-Based Bi-Fidelity Stochastic Trust Region Method for Derivative-Free Stochastic Optimization

TL;DR

This paper introduces ASTRO-BFDF, an adaptive bi-fidelity stochastic trust region method for derivative-free optimization that efficiently balances sampling costs and convergence to stationary points.

Contribution

It develops a novel adaptive sampling algorithm that leverages low-fidelity models to improve efficiency in bi-fidelity stochastic optimization without derivatives.

Findings

Proves convergence to first-order stationary points almost surely.

Demonstrates effectiveness on synthetic and simulation-based problems.

Balances Monte Carlo and bi-fidelity sampling adaptively.

Abstract

Bi-fidelity stochastic optimization has gained increasing attention as an efficient approach to reduce computational costs by leveraging a low-fidelity (LF) model to optimize an expensive high-fidelity (HF) objective. In this paper, we propose ASTRO-BFDF, an adaptive sampling trust region method specifically designed for unconstrained bi-fidelity stochastic derivative-free optimization problems. In ASTRO-BFDF, the LF function serves two purposes: (i) to identify better iterates for the HF function when the optimization process indicates a high correlation between them, and (ii) to reduce the variance of the HF function estimates using bi-fidelity Monte Carlo (BFMC). The algorithm dynamically determines sample sizes while adaptively choosing between crude Monte Carlo and BFMC to balance the trade-off between optimization and sampling errors. We prove that the iterates generated by ASTRO-BFDF converge to the first-order stationary point almost surely. Additionally, we demonstrate the effectiveness of the proposed algorithm through numerical experiments on synthetic problems and simulation optimization problems involving discrete event systems.