Combinatorial Penalties: Which structures are preserved by convex relaxations?

TL;DR

This paper investigates the effectiveness of convex relaxations for combinatorial penalties, introducing new theoretical insights and conditions for support recovery, with implications for structured regularization.

Contribution

It identifies key differences in relaxation tightness using the lower combinatorial envelope and proposes an adaptive estimator with support recovery guarantees.

Findings

Key differences in relaxation tightness are characterized.

New necessary conditions for support identification are established.

An adaptive estimator with support recovery guarantees is proposed.

Abstract

We consider the homogeneous and the non-homogeneous convex relaxations for combinatorial penalty functions defined on support sets. Our study identifies key differences in the tightness of the resulting relaxations through the notion of the lower combinatorial envelope of a set-function along with new necessary conditions for support identification. We then propose a general adaptive estimator for convex monotone regularizers, and derive new sufficient conditions for support recovery in the asymptotic setting.