Retraction-based first-order feasible methods for difference-of-convex programs with smooth inequality and simple geometric constraints

TL;DR

This paper introduces retraction-based first-order methods for difference-of-convex programs with smooth inequalities and geometric constraints, ensuring feasibility and analyzing convergence, with applications to compressed sensing.

Contribution

It develops a novel retraction-based feasible algorithm for DC programs with convergence analysis and applies it to structured compressed sensing problems.

Findings

Global subsequential convergence under convex constraints

Local linear convergence for KL exponent 1/2 functions

Effective application to group-structured compressed sensing

Abstract

In this paper, we propose first-order feasible methods for difference-of-convex (DC) programs with smooth inequality and simple geometric constraints. Our strategy for maintaining feasibility of the iterates is based on a "retraction" idea adapted from the literature of manifold optimization. When the constraints are convex, we establish the global subsequential convergence of the sequence generated by our algorithm under strict feasibility condition, and analyze its convergence rate when the objective is in addition convex according to the Kurdyka-Lojasiewicz (KL) exponent of the extended objective (i.e., sum of the objective and the indicator function of the constraint set). We also show that the extended objective of a large class of Euclidean norm (and more generally, group LASSO penalty) regularized convex optimization problems is a KL function with exponent $\frac12$; consequently, our algorithm is locally linearly convergent when applied to these problems. We then extend our method to solve DC programs with a single specially structured nonconvex constraint. Finally, we discuss how our algorithms can be applied to solve two concrete optimization problems, namely, group-structured compressed sensing problems with Gaussian measurement noise and compressed sensing problems with Cauchy measurement noise, and illustrate the empirical performance of our algorithms.