Structure-Aware Methods for Expensive Derivative-Free Nonsmooth Composite Optimization

TL;DR

This paper introduces new structure-aware methods for expensive, nonsmooth, composite optimization problems with bound constraints, including a novel manifold sampling algorithm and a more complex subproblem approach, with proven convergence and extensive testing.

Contribution

It presents two new algorithms tailored for expensive, nonsmooth composite optimization, with rigorous convergence analysis and open-source implementations.

Findings

The manifold sampling algorithm ( exttt{MSP}) converges under certain conditions.

The exttt{Goomba} method handles more difficult subproblems effectively.

Extensive testing demonstrates the practical performance of both methods.

Abstract

We present new methods for solving a broad class of bound-constrained nonsmooth composite minimization problems. These methods are specially designed for objectives that are some known mapping of outputs from a computationally expensive function. We provide accompanying implementations of these methods: in particular, a novel manifold sampling algorithm (\mspshortref) with subproblems that are in a sense primal versions of the dual problems solved by previous manifold sampling methods and a method (\goombahref) that employs more difficult optimization subproblems. For these two methods, we provide rigorous convergence analysis and guarantees. We demonstrate extensive testing of these methods. Open-source implementations of the methods developed in this manuscript can be found at \url{github.com/POptUS/IBCDFO/}.