Finding directional stationary points of DC programs

TL;DR

This paper introduces a unified approach for finding directional stationary points in non-smooth DC programs, extending existing methods and providing convergence guarantees, including a stochastic variant.

Contribution

It generalizes classical DCA to handle non-smooth components and establishes strong convergence results, including for the stochastic case.

Findings

Unified framework for non-smooth DC programs

Convergence proven under Łojasiewicz inequality

Stochastic variant with almost sure convergence

Abstract

We address the problem of computing stationary points for non-smooth, non-convex optimization problems. While this topic is well studied in the smooth setting, fewer algorithmic and theoretical results exist for the non-smooth case. Within Difference-of-Convex functions (DC) programming, the well-known DC Algorithm (DCA) is a standard method for computing critical points, whose definition depends on the chosen DC decomposition. More recently, some works have focused on computing directional stationary points - a stronger notion that does not depend on any particular DC decomposition - for specific non-smooth DC programs, where the second DC component is the pointwise maximum of finitely many smooth convex functions. In this contribution, we propose a new and unified approach for identifying directional stationary points of non-smooth DC programs, where both DC components may be non-smooth. Our framework generalizes both the classical DCA and the above recent approaches. It applies when the second DC component is the pointwise maximum of a continuous family of smooth convex functions, and more generally, to any continuous convex function on the whole space. We also establish strong convergence results. Specifically, when the second DC component is the pointwise maximum of finitely many $C^{1,1}$ - smooth convex functions, we prove under the \L ojasiewicz inequality that the entire sequence generated by our algorithm converges. This extends previous results requiring global $C^{1,1}$ - smoothness. Finally, we introduce a randomized (stochastic) variant of our method and prove its almost sure convergence, thereby extending our deterministic results to the randomized setting.