Model-Based Derivative-Free Methods for Convex-Constrained Optimization

TL;DR

This paper introduces a new model-based derivative-free optimization method for convex-constrained problems, achieving global convergence with improved model accuracy requirements and demonstrating strong practical performance.

Contribution

It develops a derivative-free optimization algorithm for convex constraints with weaker model accuracy conditions and provides a comprehensive theory for interpolation set management.

Findings

Proves global convergence and $O( ext{epsilon}^{-2})$ complexity.

Constructs interpolation models using only feasible points.

Shows strong practical performance on nonlinear least-squares problems.

Abstract

We present a model-based derivative-free method for optimization subject to general convex constraints, which we assume are unrelaxable and accessed only through a projection operator that is cheap to evaluate. We prove global convergence and a worst-case complexity of $O(\epsilon^{-2})$ iterations and objective evaluations for nonconvex functions, matching results for the unconstrained case. We introduce new, weaker requirements on model accuracy compared to existing methods. As a result, sufficiently accurate interpolation models can be constructed only using feasible points. We develop a comprehensive theory of interpolation set management in this regime for linear and composite linear models. We implement our approach for nonlinear least-squares problems and demonstrate strong practical performance compared to general-purpose solvers.