K-Dimensional Coding Schemes in Hilbert Spaces

TL;DR

This paper introduces a general coding framework for Hilbert space data using finite dimensional vectors, providing bounds on reconstruction error and connecting to methods like K-means and NMF.

Contribution

It proposes a unified empirical risk minimization approach for coding in Hilbert spaces, with theoretical error bounds and applications to existing methods.

Findings

Derived bounds on expected reconstruction error.

Unified framework encompassing K-means, NMF, and sparse coding.

Insights into the role of codebook and linear operators.

Abstract

This paper presents a general coding method where data in a Hilbert space are represented by finite dimensional coding vectors. The method is based on empirical risk minimization within a certain class of linear operators, which map the set of coding vectors to the Hilbert space. Two results bounding the expected reconstruction error of the method are derived, which highlight the role played by the codebook and the class of linear operators. The results are specialized to some cases of practical importance, including K-means clustering, nonnegative matrix factorization and other sparse coding methods.