Some limits to nonparametric estimation for ergodic processes

TL;DR

This paper presents a fundamental limitation in nonparametric estimation of binary ergodic processes, showing that no countable class of estimators can universally estimate certain zero-entropy processes accurately.

Contribution

It introduces a new negative result demonstrating the impossibility of universal nonparametric distribution estimation for all ergodic processes.

Findings

No countable estimator class can universally estimate some zero-entropy ergodic processes.

The result differs from previous negative results on universal forecasting schemes.

A related result by B. Weiss is also discussed.

Abstract

A new negative result for nonparametric distribution estimation of binary ergodic processes is shown. The problem of estimation of distribution with any degree of accuracy is studied. Then it is shown that for any countable class of estimators there is a zero-entropy binary ergodic process that is inconsistent with the class of estimators. Our result is different from other negative results for universal forecasting scheme of ergodic processes. We also introduce a related result by B. Weiss.