Strong NP-Hardness for Sparse Optimization with Concave Penalty Functions

TL;DR

This paper proves that finding near-optimal solutions for sparse optimization problems with nonconvex penalties is strongly NP-hard, indicating fundamental computational difficulty even for approximate solutions.

Contribution

It establishes the strong NP-hardness of sparse optimization with concave penalties, extending to a broad class of loss and penalty functions, and highlights computational limitations.

Findings

Strong NP-hardness for approximate solutions

Applicable to broad loss and penalty functions

Implication of computational intractability unless P=NP

Abstract

Consider the regularized sparse minimization problem, which involves empirical sums of loss functions for $n$ data points (each of dimension $d$) and a nonconvex sparsity penalty. We prove that finding an $\mathcal{O}(n^{c_1}d^{c_2})$-optimal solution to the regularized sparse optimization problem is strongly NP-hard for any $c_1, c_2\in [0,1)$ such that $c_1+c_2<1$. The result applies to a broad class of loss functions and sparse penalty functions. It suggests that one cannot even approximately solve the sparse optimization problem in polynomial time, unless P $=$ NP.