Approximation Bounds for Sparse Programs

TL;DR

This paper investigates the duality gap in sparsity-constrained optimization problems over low-dimensional spaces, providing bounds and procedures to recover feasible solutions, with implications for the tightness of relaxations as feature cardinality increases.

Contribution

It introduces data-driven bounds on the duality gap for sparse programs and an efficient primalization method to find feasible solutions, leveraging the Shapley-Folkman theorem.

Findings

Duality gap is small in low-dimensional sparse problems.

Relaxation tightness improves with larger target cardinality.

Proposed primalization procedure effectively recovers feasible points.

Abstract

We show that sparsity constrained optimization problems over low dimensional spaces tend to have a small duality gap. We use the Shapley-Folkman theorem to derive both data-driven bounds on the duality gap, and an efficient primalization procedure to recover feasible points satisfying these bounds. These error bounds are proportional to the rate of growth of the objective with the target cardinality, which means in particular that the relaxation is nearly tight as soon as the target cardinality is large enough so that only uninformative features are added.