Provable Data Clustering via Innovation Search

TL;DR

This paper analyzes the Innovation Pursuit algorithm for subspace clustering, demonstrating its robustness in highly intersected subspaces and proposing a projection technique to enhance its performance.

Contribution

It provides theoretical insights into Innovation Pursuit's effectiveness under heavy subspace intersections and introduces a projection method to improve clustering accuracy.

Findings

Innovation Pursuit performs well with heavily intersected subspaces.

New theoretical conditions require only the innovative components to be incoherent.

A projection technique boosts the algorithm's clustering performance.

Abstract

This paper studies the subspace clustering problem in which data points collected from high-dimensional ambient space lie in a union of linear subspaces. Subspace clustering becomes challenging when the dimension of intersection between subspaces is large and most of the self-representation based methods are sensitive to the intersection between the span of clusters. In sharp contrast to the self-representation based methods, a recently proposed clustering method termed Innovation Pursuit, computed a set of optimal directions (directions of innovation) to build the adjacency matrix. This paper focuses on the Innovation Pursuit Algorithm to shed light on its impressive performance when the subspaces are heavily intersected. It is shown that in contrast to most of the existing methods which require the subspaces to be sufficiently incoherent with each other, Innovation Pursuit only requires the innovative components of the subspaces to be sufficiently incoherent with each other. These new sufficient conditions allow the clusters to be strongly close to each other. Motivated by the presented theoretical analysis, a simple yet effective projection based technique is proposed which we show with both numerical and theoretical results that it can boost the performance of Innovation Pursuit.