Straight-Through meets Sparse Recovery: the Support Exploration Algorithm

TL;DR

This paper investigates the use of the straight-through estimator (STE) in sparse support recovery, introducing the Support Exploration Algorithm (SEA) which outperforms existing methods especially with coherent measurement matrices.

Contribution

The paper introduces SEA, a novel sparsity-promoting algorithm, and provides theoretical analysis of its support recovery guarantees under RIP conditions.

Findings

SEA explores more supports than state-of-the-art methods.

SEA achieves superior support recovery performance in experiments.

Recovery guarantees are established under RIP, with conditions comparable to existing methods.

Abstract

The {\it straight-through estimator} (STE) is commonly used to optimize quantized neural networks, yet its contexts of effective performance are still unclear despite empirical successes.To make a step forward in this comprehension, we apply STE to a well-understood problem: {\it sparse support recovery}. We introduce the {\it Support Exploration Algorithm} (SEA), a novel algorithm promoting sparsity, and we analyze its performance in support recovery (a.k.a. model selection) problems. SEA explores more supports than the state-of-the-art, leading to superior performance in experiments, especially when the columns of $A$ are strongly coherent.The theoretical analysis considers recovery guarantees when the linear measurements matrix $A$ satisfies the {\it Restricted Isometry Property} (RIP).The sufficient conditions of recovery are comparable but more stringent than those of the state-of-the-art in sparse support recovery. Their significance lies mainly in their applicability to an instance of the STE.