Relations Between the Conditional Normalized Maximum Likelihood Distributions and the Latent Information Priors

TL;DR

This paper explores the theoretical relationship between CNML distributions and Bayesian predictive densities based on latent information priors, revealing asymptotic and exact equivalences under certain conditions.

Contribution

It establishes a formal connection between CNML3 and latent information priors, showing their asymptotic and sometimes exact equivalence in predictive densities.

Findings

Bayes projection of CNML3 is asymptotically identical to Bayesian predictive density based on LIP.

Sum of Bayes projection divergence and conditional mutual information is asymptotically constant.

Under stronger assumptions, BPCNML3 exactly matches the Bayesian predictive density based on LIP.

Abstract

We reveal the relations between the conditional normalized maximum likelihood (CNML) distributions and Bayesian predictive densities based on the latent information priors (LIPs). In particular, CNML3, which is one type of CNML distributions, is investigated. The Bayes projection of a predictive density, which is an information projection of the predictive density on a set of Bayesian predictive densities, is considered. We prove that the sum of the Bayes projection divergence of CNML3 and the conditional mutual information is asymptotically constant. This result implies that the Bayes projection of CNML3 (BPCNML3) is asymptotically identical to the Bayesian predictive density based on LIP. In addition, under some stronger assumptions, we show that BPCNML3 exactly coincides with the Bayesian predictive density based on LIP.