A Nonconvex Proximal Splitting Algorithm under Moreau-Yosida Regularization

TL;DR

This paper introduces a new multiblock primal-dual algorithm for nonconvex, nonsmooth optimization problems involving Moreau-Yosida regularization, with proven convergence and practical effectiveness in machine learning tasks.

Contribution

It proposes a novel stabilized multiblock primal-dual algorithm for nonconvex problems with Moreau-Yosida regularization, including a comprehensive convergence analysis.

Findings

Algorithm converges under certain optimality conditions.

Effective in robust regression tasks.

Performs well in joint feature selection and semi-supervised learning.

Abstract

We tackle highly nonconvex, nonsmooth composite optimization problems whose objectives comprise a Moreau-Yosida regularized term. Classical nonconvex proximal splitting algorithms, such as nonconvex ADMM, suffer from lack of convergence for such a problem class. To overcome this difficulty, in this work we consider a lifted variant of the Moreau-Yosida regularized model and propose a novel multiblock primal-dual algorithm that intrinsically stabilizes the dual block. We provide a complete convergence analysis of our algorithm and identify respective optimality qualifications under which stationarity of the original model is retrieved at convergence. Numerically, we demonstrate the relevance of Moreau-Yosida regularized models and the efficiency of our algorithm on robust regression as well as joint feature selection and semi-supervised learning.