Sampling, Metric Entropy and Dimensionality Reduction

TL;DR

This paper establishes a probabilistic relationship between sampling entropy and Kolmogorov entropy in Hilbert spaces, demonstrating that piecewise smooth functions can be sampled efficiently, confirming the Eckhoff conjecture.

Contribution

It shows that in a probabilistic setting, sampling entropy is bounded by Kolmogorov entropy, enabling efficient sampling of piecewise smooth functions, and settles the Eckhoff conjecture.

Findings

Sampling $ ext{e}$-entropy is bounded by Kolmogorov $ ext{e}$-entropy.

Piecewise smooth functions can be sampled with the same accuracy as smooth functions.

Settles the Eckhoff conjecture for univariate piecewise $C^k$-smooth functions.

Abstract

Let $Q$ be a relatively compact subset in a Hilbert space $V$. For a given $\e>0$ let $N(\e,Q)$ be the minimal number of linear measurements, sufficient to reconstruct any $x \in Q$ with the accuracy $\e$. We call $N(\e,Q)$ a sampling $\e$-entropy of $Q$. Using Dimensionality Reduction, as provided by the Johnson-Lindenstrauss lemma, we show that, in an appropriate probabilistic setting, $N(\e,Q)$ is bounded from above by the Kolmogorov's $\e$-entropy $H(\e,Q)$, defined as $H(\e,Q)=\log M(\e,Q)$, with $M(\e,Q)$ being the minimal number of $\e$-balls covering $Q$. As the main application, we show that piecewise smooth (piecewise analytic) functions in one and several variables can be sampled with essentially the same accuracy rate as their regular counterparts. For univariate piecewise $C^k$-smooth functions this result, which settles the so-called Eckhoff conjecture, was recently established in \cite{Bat} via a deterministic "algebraic reconstruction" algorithm.