Dimensionality Reduction with Subspace Structure Preservation

TL;DR

This paper introduces a novel dimensionality reduction method that preserves the independence structure of data sampled from multiple subspaces, supported by theoretical guarantees and empirical results on synthetic and real data.

Contribution

It demonstrates that 2K projection vectors suffice to preserve independence in K-class subspace data and proposes a new algorithm based on this insight.

Findings

Achieves state-of-the-art results on synthetic data

Effectively preserves subspace independence structure

Outperforms popular dimensionality reduction techniques

Abstract

Modeling data as being sampled from a union of independent subspaces has been widely applied to a number of real world applications. However, dimensionality reduction approaches that theoretically preserve this independence assumption have not been well studied. Our key contribution is to show that $2K$ projection vectors are sufficient for the independence preservation of any $K$ class data sampled from a union of independent subspaces. It is this non-trivial observation that we use for designing our dimensionality reduction technique. In this paper, we propose a novel dimensionality reduction algorithm that theoretically preserves this structure for a given dataset. We support our theoretical analysis with empirical results on both synthetic and real world data achieving \textit{state-of-the-art} results compared to popular dimensionality reduction techniques.