Shadow splitting methods for nonconvex optimisation: epi-approximation, convergence and saddle point avoidance

TL;DR

This paper introduces a novel shadow splitting method for nonconvex optimization, providing convergence guarantees and saddle point avoidance, supported by variational analysis and numerical experiments.

Contribution

It develops the shadow Davis-Yin three-operator splitting method with convergence analysis and saddle point avoidance guarantees for nonconvex problems.

Findings

Convergence of the damped shadow method is established.

The method almost surely avoids strict saddle points.

Numerical experiments validate theoretical results.

Abstract

We propose the shadow Davis-Yin three-operator splitting method to solve nonconvex optimisation problems. Its convergence analysis is based on a merit function resembling the Moreau envelope. We explore variational analysis properties behind the merit function and the iteration operators associated with the shadow method. By capitalising on these results, we establish convergence of a damped version of the shadow method via sufficient descent of the merit function, and obtain (almost surely) guarantees of avoidance of strict saddle points of weakly convex semialgebraic optimisation problems. We perform numerical experiments for a nonconvex variable selection problem to test our findings.