Distances between nested densities and a measure of the impact of the prior in Bayesian statistics

TL;DR

This paper develops explicit bounds for Wasserstein distances between univariate distributions with nested supports, with applications to quantifying the impact of priors on Bayesian posterior distributions.

Contribution

It introduces a new Stein's method variant to derive tight bounds on Wasserstein distances, specifically applied to Bayesian prior influence analysis.

Findings

Explicit bounds for Wasserstein distance between distributions with nested supports.

Data-driven bounds on the impact of priors on posterior distributions.

Finite sample results confirming minimal prior influence with large data.

Abstract

In this paper we propose tight upper and lower bounds for the Wasserstein distance between any two {{univariate continuous distributions}} with probability densities $p_1$ and $p_2$ having nested supports. These explicit bounds are expressed in terms of the derivative of the likelihood ratio $p_1/p_2$ as well as the Stein kernel $\tau_1$ of $p_1$. The method of proof relies on a new variant of Stein's method which manipulates Stein operators. We give several applications of these bounds. Our main application is in Bayesian statistics : we derive explicit data-driven bounds on the Wasserstein distance between the posterior distribution based on a given prior and the no-prior posterior based uniquely on the sampling distribution. This is the first finite sample result confirming the well-known fact that with well-identified parameters and large sample sizes, reasonable choices of prior distributions will have only minor effects on posterior inferences if the data are benign.