Superlinear convergence in nonsmooth optimization via higher-order cutting-plane models

TL;DR

This paper demonstrates that higher-order cutting-plane models can achieve superlinear convergence in nonsmooth optimization, extending the benefits of smooth optimization techniques to nonsmooth problems.

Contribution

It introduces higher-order cutting-plane models for nonsmooth functions and develops a trust-region bundle method that attains superlinear convergence.

Findings

Superlinear convergence achieved in nonsmooth optimization with higher-order models

Constructed a trust-region bundle method with superlinear convergence

Numerical experiments confirm theoretical convergence results

Abstract

A cutting-plane model for a nonsmooth function is the maximum of several first-order expansions centered at different points. Using such a model in a bundle method leads to linear convergence (of serious steps) to a minimum. In smooth optimization, superlinear convergence can be achieved by using higher-order models. We show that the same is true for the nonsmooth case, i.e., we show that cutting-plane models involving higher-order expansions can be used to achieve superlinear convergence in nonsmooth optimization. We first formally define higher-order cutting-plane models for lower-$C^2$ functions and derive an error estimate. Afterwards, we construct a trust-region bundle method based on these models that achieves local superlinear convergence of serious steps, and overall superlinear convergence for certain finite max-type functions. Finally, we verify the superlinear convergence in numerical experiments.