Constrained stochastic blackbox optimization using a progressive barrier and probabilistic estimates

TL;DR

This paper presents StoMADS-PB, an algorithm for constrained stochastic blackbox optimization that uses probabilistic bounds and estimates to handle noisy function evaluations, extending the deterministic MADS method.

Contribution

It introduces a novel stochastic extension of the MADS algorithm with probabilistic bounds for constraint violations, enabling convergence analysis under noise.

Findings

Convergence to Clarke stationarity with probability one.

Effective handling of noisy function evaluations in constrained optimization.

The method accommodates intermediate infeasible iterates.

Abstract

This work introduces the StoMADS-PB algorithm for constrained stochastic blackbox optimization, which is an extension of the mesh adaptive direct-search (MADS) method originally developed for deterministic blackbox optimization under general constraints. The values of the objective and constraint functions are provided by a noisy blackbox, i.e., they can only be computed with random noise whose distribution is unknown. As in MADS, constraint violations are aggregated into a single constraint violation function. Since all functions values are numerically unavailable, StoMADS-PB uses estimates and introduces so-called probabilistic bounds for the violation. Such estimates and bounds obtained from stochastic observations are required to be accurate and reliable with high but fixed probabilities. The proposed method, which allows intermediate infeasible iterates, accepts new points using sufficient decrease conditions and imposing a threshold on the probabilistic bounds. Using Clarke nonsmooth calculus and martingale theory, Clarke stationarity convergence results for the objective and the violation function are derived with probability one.