Asymptotic Properties of Bayesian Predictive Densities When the Distributions of Data and Target Variables are Different

TL;DR

This paper investigates the asymptotic behavior of Bayesian predictive densities when the data and target variables have different distributions, using information geometry and introducing a new predictive metric.

Contribution

It introduces a novel predictive metric based on Fisher information matrices and demonstrates the asymptotic dominance of certain Bayesian predictive densities.

Findings

Predictive densities based on superharmonic priors outperform volume element priors.

The framework extends information geometry to settings with different data and target distributions.

Asymptotic properties are characterized using the new predictive metric.

Abstract

Bayesian predictive densities when the observed data $x$ and the target variable $y$ to be predicted have different distributions are investigated by using the framework of information geometry. The performance of predictive densities is evaluated by the Kullback--Leibler divergence. The parametric models are formulated as Riemannian manifolds. In the conventional setting in which $x$ and $y$ have the same distribution, the Fisher--Rao metric and the Jeffreys prior play essential roles. In the present setting in which $x$ and $y$ have different distributions, a new metric, which we call the predictive metric, constructed by using the Fisher information matrices of $x$ and $y$, and the volume element based on the predictive metric play the corresponding roles. It is shown that Bayesian predictive densities based on priors constructed by using non-constant positive superharmonic functions with respect to the predictive metric asymptotically dominate those based on the volume element prior of the predictive metric.