Solving Sparsity Constrained PCA, Regression, and QCQP via the Spartrahedron

TL;DR

This paper introduces a novel convex relaxation framework using the spartrahedron for solving sparsity-constrained QCQPs, enabling efficient and certifiable solutions for problems like sparse PCA and regression.

Contribution

A new convex cone called the spartrahedron is proposed, providing tight SDP relaxations for sparsity-constrained QCQPs with theoretical guarantees and practical effectiveness.

Findings

SDP relaxation is tight for rank-one solutions.

The method provides approximation bounds for sparse PCA and regression.

Numerical experiments confirm practical success across multiple problems.

Abstract

Sparsity is a fundamental modeling principle in statistics, signal processing, and data science. However, optimization with sparsity constraints is notoriously difficult. We introduce a new convex relaxation framework for {sparse quadratically constrained quadratic programs} (QCQPs), a class that subsumes sparse regression, sparse principal component analysis (PCA), and related problems. Our approach is based on a novel convex cone, the spartrahedron, which exactly characterizes sparsity at the matrix level. This leads to a semidefinite programming (SDP) relaxation that is tight whenever its solution is rank-one, providing a simple certificate of global optimality. We establish theoretical guarantees, including approximation bounds and exactness regions for sparse PCA and sparse ridge regression, as well as a general stability result under perturbations. Numerical experiments on sparse PCA, sparse regression, RIP constant estimation, and sparse canonical correlation analysis (CCA) demonstrate the practical success of our methods.