Leave-One-Out Cross-Validation for Bayesian Model Comparison in Large Data

TL;DR

This paper introduces an efficient method for Bayesian model comparison on large datasets by combining approximate leave-one-out surrogates with exact subsampling, significantly improving efficiency and comparison accuracy.

Contribution

It proposes a novel combination of approximate and exact LOO methods for scalable and more effective Bayesian model comparison in large data settings.

Findings

Method is significantly more efficient than previous approaches.

Improves accuracy in model comparison tasks.

Provides theoretical proofs for scalability.

Abstract

Recently, new methods for model assessment, based on subsampling and posterior approximations, have been proposed for scaling leave-one-out cross-validation (LOO) to large datasets. Although these methods work well for estimating predictive performance for individual models, they are less powerful in model comparison. We propose an efficient method for estimating differences in predictive performance by combining fast approximate LOO surrogates with exact LOO subsampling using the difference estimator and supply proofs with regards to scaling characteristics. The resulting approach can be orders of magnitude more efficient than previous approaches, as well as being better suited to model comparison.