Convergence and Recovery Guarantees of the K-Subspaces Method for Subspace Clustering

TL;DR

This paper provides convergence guarantees and a new initialization method for the K-subspaces algorithm in subspace clustering, demonstrating rapid convergence and accurate recovery under a semi-random data model.

Contribution

The paper introduces a local convergence analysis, recovery guarantees, and a spectral initialization method for the K-subspaces algorithm in subspace clustering.

Findings

Converges at superlinear rate within Θ(log log N) iterations

Spectral initialization produces a good starting point for convergence

Numerical results support theoretical guarantees

Abstract

The K-subspaces (KSS) method is a generalization of the K-means method for subspace clustering. In this work, we present local convergence analysis and a recovery guarantee for KSS, assuming data are generated by the semi-random union of subspaces model, where $N$ points are randomly sampled from $K \ge 2$ overlapping subspaces. We show that if the initial assignment of the KSS method lies within a neighborhood of a true clustering, it converges at a superlinear rate and finds the correct clustering within $\Theta(\log\log N)$ iterations with high probability. Moreover, we propose a thresholding inner-product based spectral method for initialization and prove that it produces a point in this neighborhood. We also present numerical results of the studied method to support our theoretical developments.