The Sparse(st) Optimization Problem: Reformulations, Optimality, Stationarity, and Numerical Results

TL;DR

This paper reformulates the challenging sparse optimization problem with nonlinear constraints into smooth programs, establishes stationarity and optimality conditions, and demonstrates the effectiveness of specialized Newton-type methods through numerical experiments.

Contribution

It introduces two equivalent smooth reformulations of the sparse optimization problem with nonlinear constraints and develops tailored stationarity and second-order conditions.

Findings

Reformulations are equivalent in terms of minima.

Lagrange-Newton methods are locally fast convergent.

Numerical results show improved sparse solutions.

Abstract

We consider the sparse optimization problem with nonlinear constraints and an objective function, which is given by the sum of a general smooth mapping and an additional term defined by the $ \ell_0 $-quasi-norm. This term is used to obtain sparse solutions, but difficult to handle due to its nonconvexity and nonsmoothness (the sparsity-improving term is even discontinuous). The aim of this paper is to present two reformulations of this program as a smooth nonlinear program with complementarity-type constraints. We show that these programs are equivalent in terms of local and global minima and introduce a problem-tailored stationarity concept, which turns out to coincide with the standard KKT conditions of the two reformulated problems. In addition, a suitable constraint qualification as well as second-order conditions for the sparse optimization problem are investigated. These are then used to show that three Lagrange-Newton-type methods are locally fast convergent. Numerical results on different classes of test problems indicate that these methods can be used to drastically improve sparse solutions obtained by some other (globally convergent) methods for sparse optimization problems.