Efficient DC Algorithm for Constrained Sparse Optimization

TL;DR

This paper introduces an efficient DC algorithm for constrained sparse optimization that leverages a new DC representation of the $ ext{ extlbrackdbl}0 ext{ extbrackdbl}$-constraint, enabling faster solutions with additional convex constraints.

Contribution

The paper proposes a novel DC reformulation allowing PDCA to efficiently handle convex constraints, and introduces an accelerated version with optimal convergence rates.

Findings

The proposed APDCA achieves faster convergence in numerical experiments.

The reformulation simplifies subproblems to projection operations, improving efficiency.

APDCA attains the optimal convergence rate for convex problems.

Abstract

We address the minimization of a smooth objective function under an $\ell_0$-constraint and simple convex constraints. When the problem has no constraints except the $\ell_0$-constraint, some efficient algorithms are available; for example, Proximal DC (Difference of Convex functions) Algorithm (PDCA) repeatedly evaluates closed-form solutions of convex subproblems, leading to a stationary point of the $\ell_0$-constrained problem. However, when the problem has additional convex constraints, they become inefficient because it is difficult to obtain closed-form solutions of the associated subproblems. In this paper, we reformulate the problem by employing a new DC representation of the $\ell_0$-constraint, so that PDCA can retain the efficiency by reducing its subproblems to the projection operation onto a convex set. Moreover, inspired by the Nesterov's acceleration technique for proximal methods, we propose the Accelerated PDCA (APDCA), which attains the optimal convergence rate if applied to convex programs, and performs well in numerical experiments.