Sparse Subspace Clustering: Algorithm, Theory, and Applications

TL;DR

This paper introduces Sparse Subspace Clustering (SSC), an algorithm for clustering high-dimensional data lying near multiple low-dimensional subspaces, capable of handling noise, outliers, and missing data, with theoretical guarantees and practical success.

Contribution

The paper proposes a convex relaxation of sparse subspace clustering, providing theoretical analysis and demonstrating effectiveness on real-world datasets.

Findings

Successfully clusters data in synthetic and real-world experiments.

Handles noise, outliers, and missing data effectively.

Theoretically guarantees correct subspace recovery under certain conditions.

Abstract

In many real-world problems, we are dealing with collections of high-dimensional data, such as images, videos, text and web documents, DNA microarray data, and more. Often, high-dimensional data lie close to low-dimensional structures corresponding to several classes or categories the data belongs to. In this paper, we propose and study an algorithm, called Sparse Subspace Clustering (SSC), to cluster data points that lie in a union of low-dimensional subspaces. The key idea is that, among infinitely many possible representations of a data point in terms of other points, a sparse representation corresponds to selecting a few points from the same subspace. This motivates solving a sparse optimization program whose solution is used in a spectral clustering framework to infer the clustering of data into subspaces. Since solving the sparse optimization program is in general NP-hard, we consider a convex relaxation and show that, under appropriate conditions on the arrangement of subspaces and the distribution of data, the proposed minimization program succeeds in recovering the desired sparse representations. The proposed algorithm can be solved efficiently and can handle data points near the intersections of subspaces. Another key advantage of the proposed algorithm with respect to the state of the art is that it can deal with data nuisances, such as noise, sparse outlying entries, and missing entries, directly by incorporating the model of the data into the sparse optimization program. We demonstrate the effectiveness of the proposed algorithm through experiments on synthetic data as well as the two real-world problems of motion segmentation and face clustering.