Statistical estimation requires unbounded memory

TL;DR

This paper proves that bounded-memory estimators cannot consistently estimate certain statistical functionals, and introduces bounded-memory approximation techniques using automata theory and stochastic processes.

Contribution

It establishes the impossibility of bounded-memory consistent estimators for some functionals and proposes new approximation methods leveraging automata and stochastic processes.

Findings

Bounded-memory estimators cannot consistently estimate certain functionals.

Proposed bounded-memory approximation techniques using automata theory.

Raised questions for future research on bounded-memory estimation.

Abstract

We investigate the existence of bounded-memory consistent estimators of various statistical functionals. This question is resolved in the negative in a rather strong sense. We propose various bounded-memory approximations, using techniques from automata theory and stochastic processes. Some questions of potential interest are raised for future work.