On A Necessary Condition For Posterior Inconsistency: New Insights From A Classic Counterexample

TL;DR

This paper investigates the conditions under which Bayesian posterior distributions in density estimation become inconsistent, revealing that inconsistency requires implausibly precise prior knowledge and is thus generally unlikely in realistic scenarios.

Contribution

It identifies a necessary condition for posterior inconsistency, showing that such inconsistency only occurs under highly pathological priors with unrealistic assumptions.

Findings

Inconsistency requires persistent concentration on densities with high likelihood ratios.

Such behavior demands priors encoding implausibly precise knowledge of the true distribution.

Posterior inconsistency is essentially unattainable in realistic inference problems.

Abstract

The consistency of posterior distributions in density estimation is at the core of Bayesian statistical theory. Classical work established sufficient conditions, typically combining KL support with complexity bounds on sieves of high prior mass, to guarantee consistency with respect to the Hellinger distance. Yet no systematic theory explains a widely held belief: under KL support, Hellinger consistency is exceptionally hard to violate. This suggests that existing sufficient conditions, while useful in practice, may overlook some key aspects of posterior behavior. We address this gap by directly investigating what must fail for inconsistency to arise, aiming to identify a substantive necessary condition for Hellinger inconsistency. Our starting point is Andrew Barron's classical counterexample, the only known violation of Hellinger consistency under KL support, which relies on a contrived family of oscillatory densities and a prior with atoms. We show that, within a broad class of models including Barron's, inconsistency requires persistent posterior concentration on densities with exponentially high likelihood ratios. In turn, such behavior demands a prior encoding implausibly precise knowledge of the true, yet unknown data-generating distribution, making inconsistency essentially unattainable in any realistic inference problem. Our results confirm the long-standing intuition that posterior inconsistency in density estimation is not a natural phenomenon, but rather an artifact of pathological prior constructions.