Laws of Large Numbers for Information Resolution

TL;DR

This paper develops a framework for understanding how independent samples can asymptotically recover the entire information structure of a probability space, with applications to stochastic embedding and random forests.

Contribution

It introduces a novel approach to establish laws of large numbers for the recovery of the sigma-field of a probability space, extending classical results to information resolution.

Findings

The Borel sigma-field in  can be recovered under iid sampling.

Finite-sample L^1 bounds for sigma-field convergence on [0,1]^d.

Applications include solutions to the Skorokhod embedding problem and analysis of random forest loss.

Abstract

Laws of large numbers establish asymptotic guarantees for recovering features of a probability distribution using independent samples. We introduce a framework for proving analogous results for recovery of the $\sigma$-field of a probability space, interpreted as information resolution--the granularity of measurable events given by comparison to our samples. Our main results show that, under iid sampling, the Borel $\sigma$-field in $\mathbb R^d$ and in more general metric spaces can be recovered in the strongest possible mode of convergence. We also derive finite-sample $L^1$ bounds for uniform convergence of $\sigma$-fields on $[0,1]^d$. We illustrate the use of our framework with two applications: constructing randomized solutions to the Skorokhod embedding problem, and analyzing the loss of variants of random forests for regression.