BISTRO -- A Bi-Fidelity Stochastic Gradient Framework using Trust-Regions for Optimization Under Uncertainty

TL;DR

BISTRO is a novel bi-fidelity stochastic trust-region method that accelerates optimization under uncertainty by combining curvature-based local search with variance-reduced stochastic gradient descent, achieving faster convergence and reduced computational costs.

Contribution

It introduces a new bi-fidelity optimization framework that exploits both curvature and correlation, with convergence guarantees and practical efficiency improvements.

Findings

BISTRO converges faster than existing methods on benchmark problems.

It reduces computational expense by up to 29 times.

The method guarantees convergence with an optimal rate under certain conditions.

Abstract

Stochastic optimization of engineering systems is often infeasible due to repeated evaluations of a computationally expensive, high-fidelity simulation. Bi-fidelity methods mitigate this challenge by leveraging a cheaper, approximate model to accelerate convergence. Most existing bi-fidelity approaches, however, exploit either design-space curvature or random-space correlation, not both. We present BISTRO - a BI-fidelity Stochastic Trust-Region Optimizer for unconstrained optimization under uncertainty through a stochastic approximation procedure. This approach exploits the curvature information of a low-fidelity objective function to converge within a basin of a local minimum of the high-fidelity model where low-fidelity curvature information is no longer valuable. The method then switches to a variance-reduced stochastic gradient descent procedure. We provide convergence guarantees in expectation under certain regularity assumptions and ensure the best-case $\mathcal{O}(1/n)$ convergence rate for stochastic optimization. On benchmark problems and a 20-dimensional space shuttle reentry case, BISTRO converges faster than adaptive sampling and variance reduction procedures and cuts computational expense by up to 29x.