A Unifying Framework for Sparsity Constrained Optimization

TL;DR

This paper introduces a unifying optimization framework for problems with convex and sparsity constraints, providing new optimality conditions, an algorithm, and empirical comparisons with existing methods.

Contribution

It develops a novel optimality condition based on a tailored neighborhood, unifies existing conditions, and proposes an algorithm with proven convergence for sparsity-constrained optimization.

Findings

The proposed algorithm converges to stationary points satisfying the new optimality condition.

The framework can recover existing optimality conditions as special cases.

Computational experiments show the framework's effectiveness compared to state-of-the-art methods.

Abstract

In this paper, we consider the optimization problem of minimizing a continuously differentiable function subject to both convex constraints and sparsity constraints. By exploiting a mixed-integer reformulation from the literature, we define a necessary optimality condition based on a tailored neighborhood that allows to take into account potential changes of the support set. We then propose an algorithmic framework to tackle the considered class of problems and prove its convergence to points satisfying the newly introduced concept of stationarity. We further show that, by suitably choosing the neighborhood, other well-known optimality conditions from the literature can be recovered at the limit points of the sequence produced by the algorithm. Finally, we analyze the computational impact of the neighborhood size within our framework and in the comparison with some state-of-the-art algorithms, namely, the Penalty Decomposition method and the Greedy Sparse-Simplex method. The algorithms have been tested using a benchmark related to sparse logistic regression problems.