Convergence of Pseudo-Bayes Factors in Forward and Inverse Regression Problems

TL;DR

This paper proves the almost sure exponential convergence of pseudo-Bayes factors in large samples for dependent data and model misspecifications, focusing on regression problems in forward and inverse contexts, with theoretical and simulation insights.

Contribution

It establishes the convergence properties of pseudo-Bayes factors in complex regression settings, extending their theoretical understanding and practical applicability.

Findings

Pseudo-Bayes factors converge exponentially almost surely in large samples.

Simulation studies demonstrate effectiveness in small sample Poisson, logit, and probit regressions.

Variable selection benefits are illustrated in both linear and nonparametric models.

Abstract

In the Bayesian literature on model comparison, Bayes factors play the leading role. In the classical statistical literature, model selection criteria are often devised used cross-validation ideas. Amalgamating the ideas of Bayes factor and cross-validation Geisser and Eddy (1979) created the pseudo-Bayes factor. The usage of cross-validation inculcates several theoretical advantages, computational simplicity and numerical stability in Bayes factors as the marginal density of the entire dataset is replaced with products of cross-validation densities of individual data points. However, the popularity of pseudo-Bayes factors is still negligible in comparison with Bayes factors, with respect to both theoretical investigations and practical applications. In this article, we establish almost sure exponential convergence of pseudo-Bayes factors for large samples under a general setup consisting of dependent data and model misspecifications. We particularly focus on general parametric and nonparametric regression setups in both forward and inverse contexts. We illustrate our theoretical results with various examples, providing explicit calculations. We also supplement our asymptotic theory with simulation experiments in small sample situations of Poisson log regression and geometric logit and probit regression, additionally addressing the variable selection problem. We consider both linear and nonparametric regression modeled by Gaussian processes for our purposes. Our simulation results provide quite interesting insights into the usage of pseudo-Bayes factors in forward and inverse setups.