Optimal Non-Linear Models for Sparsity and Sampling

TL;DR

This paper establishes the existence of optimal collections of subspaces for sparse data representation and sampling, providing theoretical foundations and an iterative algorithm applicable to finite and infinite dimensional spaces.

Contribution

It introduces a general framework for optimal subspace collections that minimize representation error and proposes an iterative method for their computation.

Findings

Proves existence of optimal subspace collections for data in Hilbert spaces.

Develops an iterative algorithm to find these optimal subspaces.

Connects the results to compressed sensing, dictionary design, and signal modeling.

Abstract

Given a set of vectors (the data) in a Hilbert space H, we prove the existence of an optimal collection of subspaces minimizing the sum of the square of the distances between each vector and its closest subspace in the collection. This collection of subspaces gives the best sparse representation for the given data, in a sense defined in the paper, and provides an optimal model for sampling in union of subspaces. The results are proved in a general setting and then applied to the case of low dimensional subspaces of R^N and to infinite dimensional shift-invariant spaces in L^2(R^d). We also present an iterative search algorithm for finding the solution subspaces. These results are tightly connected to the new emergent theories of compressed sensing and dictionary design, signal models for signals with finite rate of innovation, and the subspace segmentation problem.