Achieving Directional-Stationarity from a Single Random Direction Step

TL;DR

This paper proposes a simple random direction exploration step that, when added to existing optimization methods, guarantees convergence to directional stationary points in constrained nonsmooth nonconvex problems.

Contribution

It introduces a single exploration step that ensures all accumulation points are d-stationary without altering the convergence rates of the base method.

Findings

Guarantees all accumulation points are d-stationary almost surely.

Preserves convergence rates of the underlying optimization method.

Numerical experiments demonstrate the effectiveness of the exploration step.

Abstract

This paper addresses the challenge of obtaining strong optimality guarantees in constrained nonsmooth nonconvex optimization under mild regularity conditions, namely local Lipschitz continuity and existence and continuity of directional derivatives. While standard methods typically ensure weak stationarity notions, achieving directional (d-)stationarity remains nontrivial. We show that a random direction exploration step is sufficient to attain d-stationarity. The proposed approach augments any base optimization method with a single exploration step that samples a direction and step size and accepts the candidate based on a function value comparison. The resulting scheme guarantees that all accumulation points are d-stationary almost surely, independently of the behavior of the underlying method. Moreover, it preserves convergence rates of the base method, as established for DCA and prox-linear-type schemes. The theoretical results are complemented by numerical experiments illustrating the effect and guarantees of the exploration step.