A proximal subgradient algorithm with extrapolation for structured nonconvex nonsmooth problems

TL;DR

This paper introduces a novel extrapolated proximal subgradient algorithm designed for structured nonconvex nonsmooth optimization problems, with proven convergence properties and promising numerical results in machine learning applications.

Contribution

It proposes a new flexible algorithm with convergence guarantees for a broad class of nonconvex nonsmooth problems, extending existing methods with extrapolation techniques.

Findings

Algorithm converges subsequentially to stationary points.

Global convergence established under Kurdyka-Lojasiewicz property.

Numerical experiments show effective performance on nonconvex models.

Abstract

In this paper, we consider a class of structured nonconvex nonsmooth optimization problems, in which the objective function is formed by the sum of a possibly nonsmooth nonconvex function and a differentiable function whose gradient is Lipschitz continuous, subtracted by a weakly convex function. This type of structured problems has many practical applications in machine learning and statistics such as compressed sensing, signal recovery, sparse dictionary learning, clustering, matrix factorization, and others. We develop a flexible extrapolated proximal subgradient algorithm for solving these problems with guaranteed subsequential convergence to a stationary point. The global convergence of the whole sequence generated by our algorithm is also established under the Kurdyka-Lojasiewicz property. To illustrate the promising numerical performance of the proposed algorithm, we conduct numerical experiments on two important nonconvex models. This includes a least squares problem with a nonconvex regularization and an optimal power flow problem with distributed energy resources.