The Bernstein-von Mises theorem for the Pitman-Yor process of nonnegative type

TL;DR

This paper establishes a Bernstein-von Mises theorem for the Pitman-Yor process with nonnegative type, showing its asymptotic behavior and bias correction for valid Bayesian inference in discrete data settings.

Contribution

It derives the distributional limit of the Pitman-Yor posterior for nonnegative type, including bias correction and analysis of the effect of estimating the type parameter.

Findings

Posterior distribution converges to a normal distribution under certain conditions.

Bias correction improves the validity of credible sets for discrete distributions.

Estimating the type parameter affects the asymptotic properties and coverage of credible sets.

Abstract

The Pitman-Yor process is a random probability distribution, that can be used as a prior distribution in a nonparametric Bayesian analysis. The process is of species sampling type and generates discrete distributions, which yield of the order $n^\sigma$ different values ("species") in a random sample of size $n$, if the type $\sigma$ is positive. Thus this type parameter can be set to target true distributions of various levels of discreteness, making the Pitman-Yor process an interesting prior in this case. It was previously shown that the resulting posterior distribution is consistent if and only if the true distribution of the data is discrete. In this paper we derive the distributional limit of the posterior distribution, in the form of a (corrected) Bernstein-von Mises theorem, which previously was known only in the continuous, inconsistent case. It turns out that the Pitman-Yor posterior distribution has good behaviour if the true distribution of the data is discrete with atoms that decrease not too slowly. Credible sets derived from the posterior distribution provide valid frequentist confidence sets in this case. For a general discrete distribution, the posterior distribution, although consistent, may contain a bias which does not converge to zero at the $\sqrt{n}$ rate and invalidates posterior inference. We propose a bias correction that solves this problem. We also consider the effect of estimating the type parameter from the data, both by empirical Bayes and full Bayes methods. In a small simulation study we illustrate that without bias correction the coverage of credible sets can be arbitrarily low, also for some discrete distributions.