A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints

TL;DR

This paper introduces a proximal-gradient algorithm for constrained regularized optimization, providing theoretical guarantees and demonstrating promising numerical results on test problems.

Contribution

It presents a novel proximal-gradient method with decomposition and merit function strategies, along with convergence analysis and practical performance evaluation.

Findings

Establishes worst-case iteration complexity.

Proves convergence to first-order KKT points.

Shows effective manifold and active-set identification.

Abstract

We propose, analyze, and test a proximal-gradient method for solving regularized optimization problems with general constraints. The method employs a decomposition strategy to compute trial steps and uses a merit function to determine step acceptance or rejection. Under various assumptions, we establish a worst-case iteration complexity result, prove that limit points are first-order KKT points, and show that manifold identification and active-set identification properties hold. Preliminary numerical experiments on a subset of the CUTEst test problems and sparse canonical correlation analysis problems demonstrate the promising performance of our approach.