Uniform rates of the Glivenko-Cantelli convergence and their use in approximating Bayesian inferences

TL;DR

This paper establishes uniform convergence rates for empirical measures in Wasserstein distance, improving bounds for Bayesian and frequentist approximations, with applications to Gaussian and exponential family distributions.

Contribution

It provides new uniform bounds on Wasserstein distances for empirical measures, enhancing understanding of Bayesian and frequentist approximation quality with explicit convergence rates.

Findings

Uniform bounds for Wasserstein distances with respect to sample size n

Extension of results to Gaussian and exponential family distributions

Application to Bayesian posterior and predictive distribution approximation

Abstract

This paper deals with the problem of quantifying the approximation a probability measure by means of an empirical (in a wide sense) random probability measure, depending on the first n terms of a sequence of random elements. In Section 2, one studies the range of oscillation near zero of the Wasserstein distance $\ud^{(p)}_{\pms}$ between $\pfrak_0$ and $\hat{\pfrak}_n$, assuming that the $\xitil_i$'s are i.i.d. with $\pfrak_0$ as common law. Theorem 2.3 deals with the case in which $\pfrak_0$ is fixed as a generic element of the space of all probability measures on $(\rd, \mathscr{B}(\rd))$ and $\hat{\pfrak}_n$ coincides with the empirical measure. In Theorem 2.4 (Theorem 2.5, respectively) \pfrak_0 is a d-dimensional Gaussian distribution (an element of a distinguished type of statistical exponential family, respectively) and $\hat{\pfrak}_n$ is another $d$-dimensional Gaussian distribution with estimated mean and covariance matrix (another element of the same family with an estimated parameter, respectively). These new results improve on allied recent works (see, e.g., [31]) since they also provide uniform bounds with respect to $n$, meaning that the finiteness of the p-moment of the random variable $\sup_{n \geq 1} b_n \ud^{(p)}_{\pms}(\pfrak_0, \hat{\pfrak}_n)$ is proved for some suitable diverging sequence b_n of positive numbers. In Section 3, under the hypothesis that the $\xitil_i$'s are exchangeable, one studies the range of the random oscillation near zero of the Wasserstein distance between the conditional distribution--also called posterior--of the directing measure of the sequence, given $\xitil_1, \dots, \xitil_n$, and the point mass at $\hat{\pfrak}_n$. In a similar vein, a bound for the approximation of predictive distributions is given. Finally, Theorems from 3.3 to 3.5 reconsider Theorems from 2.3 to 2.5, respectively, according to a Bayesian perspective.