Bayes Model Selection with Path Sampling: Factor Models and Other Examples

TL;DR

This paper justifies the use of Path Sampling for factor model selection, identifies flaws in standard MCMC approaches, and introduces a new Small Change Path Sampling method that improves accuracy and model selection reliability.

Contribution

The paper provides a theoretical justification for Path Sampling in factor models, identifies issues with MCMC calculations, and proposes a new Small Change Path Sampling method that enhances estimation accuracy.

Findings

PS-SC outperforms standard Path Sampling in estimating Bayes factors.

PS-SC more accurately identifies the true, more complex factor model.

New diagnostics support the effectiveness of the proposed method.

Abstract

We prove a theorem justifying the regularity conditions which are needed for Path Sampling in Factor Models. We then show that the remaining ingredient, namely, MCMC for calculating the integrand at each point in the path, may be seriously flawed, leading to wrong estimates of Bayes factors. We provide a new method of Path Sampling (with Small Change) that works much better than standard Path Sampling in the sense of estimating the Bayes factor better and choosing the correct model more often. When the more complex factor model is true, PS-SC is substantially more accurate. New MCMC diagnostics is provided for these problems in support of our conclusions and recommendations. Some of our ideas for diagnostics and improvement in computation through small changes should apply to other methods of computation of the Bayes factor for model selection.