Compositions of Convex Functions and Fully Linear Models

TL;DR

This paper investigates error bounds in derivative-free optimization when using models for compositions of convex and smooth functions, extending the fully linear model concept to broader classes of functions.

Contribution

It introduces error bounds for compositions of convex lower semi-continuous functions with smooth vector functions, relaxing the smoothness requirement on the objective.

Findings

Derived error bounds for function value approximations.

Established bounds for subgradient vector approximations.

Extended fully linear model applicability to convex compositions.

Abstract

Derivative-free optimization (DFO) is the mathematical study of the optimization algorithms that do not use derivatives. One branch of DFO focuses on model-based DFO methods, where an approximation of the objective function is used to guide the optimization algorithm. Proving convergence of such methods often applies an assumption that the approximations form {\em fully linear models} -- an assumption that requires the true objective function to be smooth. However, some recent methods have loosened this assumption and instead worked with functions that are compositions of smooth functions with simple convex functions (the max-function or the $\ell_1$ norm). In this paper, we examine the error bounds resulting from the composition of a convex lower semi-continuous function with a smooth vector-valued function when it is possible to provide fully linear models for each component of the vector-valued function. We derive error bounds for the resulting function values and subgradient vectors.