Memory Efficient And Minimax Distribution Estimation Under Wasserstein Distance Using Bayesian Histograms

TL;DR

This paper introduces a memory-efficient Bayesian histogram method for distribution estimation under Wasserstein distance, achieving minimax optimality with significantly fewer bins and reduced memory footprint compared to traditional methods.

Contribution

The paper demonstrates that for dimensions less than twice the Wasserstein order, Bayesian histograms require fewer bins to attain minimax rates, improving memory efficiency over existing procedures.

Findings

Memory efficiency property for histograms when d < 2v.

Reduction in memory footprint by polynomial factor in sample size n.

Super-linear construction of posterior mean histogram and posterior.

Abstract

We study Bayesian histograms for distribution estimation on $[0,1]^d$ under the Wasserstein $W_v, 1 \leq v < \infty$ distance in the i.i.d sampling regime. We newly show that when $d < 2v$, histograms possess a special \textit{memory efficiency} property, whereby in reference to the sample size $n$, order $n^{d/2v}$ bins are needed to obtain minimax rate optimality. This result holds for the posterior mean histogram and with respect to posterior contraction: under the class of Borel probability measures and some classes of smooth densities. The attained memory footprint overcomes existing minimax optimal procedures by a polynomial factor in $n$; for example an $n^{1 - d/2v}$ factor reduction in the footprint when compared to the empirical measure, a minimax estimator in the Borel probability measure class. Additionally constructing both the posterior mean histogram and the posterior itself can be done super--linearly in $n$. Due to the popularity of the $W_1,W_2$ metrics and the coverage provided by the $d < 2v$ case, our results are of most practical interest in the $(d=1,v =1,2), (d=2,v=2), (d=3,v=2)$ settings and we provide simulations demonstrating the theory in several of these instances.