Least Sparsity of $p$-norm based Optimization Problems with $p > 1$

TL;DR

This paper characterizes the sparsity properties of $p$-norm optimization problems with $p > 1$, showing that solutions are generally fully supported and thus minimally sparse, impacting applications in data analytics and statistics.

Contribution

It provides a systematic analysis demonstrating that for $p > 1$, solutions to these optimization problems typically have full support, revealing their minimal sparsity and broad implications.

Findings

Optimal solutions attain full support for almost all measurement matrices.

Solutions are less sparse compared to $0 < p \, extless \, 1$ cases.

Implications for robustness and computational analysis of $p$-norm problems.

Abstract

Motivated by $\ell_p$-optimization arising from sparse optimization, high dimensional data analytics and statistics, this paper studies sparse properties of a wide range of $p$-norm based optimization problems with $p > 1$, including generalized basis pursuit, basis pursuit denoising, ridge regression, and elastic net. It is well known that when $p > 1$, these optimization problems lead to less sparse solutions. However, the quantitative characterization of the adverse sparse properties is not available. In this paper, by exploiting optimization and matrix analysis techniques, we give a systematic treatment of a broad class of $p$-norm based optimization problems for a general $p > 1$ and show that optimal solutions to these problems attain full support, and thus have the least sparsity, for almost all measurement matrices and measurement vectors. Comparison to $\ell_p$-optimization with $0 < p \le 1$ and implications to robustness are also given. These results shed light on analysis and computation of general $p$-norm based optimization problems in various applications.