Sparse Subspace Clustering via Two-Step Reweighted L1-Minimization: Algorithm and Provable Neighbor Recovery Rates

TL;DR

This paper introduces a two-step reweighted L1-minimization method for sparse subspace clustering that improves neighbor recovery accuracy by leveraging prior information and theoretical analysis under semi-random models.

Contribution

It proposes a novel weighted LASSO approach for SSC, extending previous algorithms without added complexity, and provides theoretical neighbor recovery guarantees.

Findings

Weighted LASSO improves neighbor identification accuracy.

Data weighting enhances the probability of correct neighbor recovery.

Theoretical bounds validate the effectiveness of the proposed method.

Abstract

Sparse subspace clustering (SSC) relies on sparse regression for accurate neighbor identification. Inspired by recent progress in compressive sensing, this paper proposes a new sparse regression scheme for SSC via two-step reweighted $\ell_1$-minimization, which also generalizes a two-step $\ell_1$-minimization algorithm introduced by E. J. Cand\`es et al in [The Annals of Statistics, vol. 42, no. 2, pp. 669-699, 2014] without incurring extra algorithmic complexity. To fully exploit the prior information offered by the computed sparse representation vector in the first step, our approach places a weight on each component of the regression vector, and solves a weighted LASSO in the second step. We propose a data weighting rule suitable for enhancing neighbor identification accuracy. Then, under the formulation of the dual problem of weighted LASSO, we study in depth the theoretical neighbor recovery rates of the proposed scheme. Specifically, an interesting connection between the locations of nonzeros of the optimal sparse solution to the weighted LASSO and the indexes of the active constraints of the dual problem is established. Afterwards, under the semi-random model, analytic probability lower/upper bounds for various neighbor recovery events are derived. Our analytic results confirm that, with the aid of data weighting and if the prior neighbor information is enough accurate, the proposed scheme with a higher probability can produce many correct neighbors and few incorrect neighbors as compared to the solution without data weighting. Computer simulations are provided to validate our analytic study and evidence the effectiveness of the proposed approach.