Stratified adaptive sampling for derivative-free stochastic trust-region optimization

TL;DR

This paper introduces stratified adaptive sampling strategies within trust-region algorithms to improve efficiency and reduce sample complexity in derivative-free stochastic optimization, especially for high-dimensional problems.

Contribution

It develops stratified adaptive sampling methods for trust-region algorithms, achieving lower sample complexity and better efficiency in stochastic optimization.

Findings

Reduced sample complexity demonstrated theoretically.

Numerical tests confirm superior efficiency.

Effective in high-dimensional settings.

Abstract

There is emerging evidence that trust-region (TR) algorithms are very effective at solving derivative-free nonconvex stochastic optimization problems in which the objective function is a Monte Carlo (MC) estimate. A recent strand of methodologies adaptively adjusts the sample size of the MC estimates by keeping the estimation error below a measure of stationarity induced from the TR radius. In this work we explore stratified adaptive sampling strategies to equip the TR framework with accurate estimates of the objective function, thus optimizing the required number of MC samples to reach a given {\epsilon}-accuracy of the solution. We prove a reduced sample complexity, confirm a superior efficiency via numerical tests and applications, and explore inexpensive implementations in high dimension.