Approximate cross-validation formula for Bayesian linear regression

TL;DR

This paper introduces an approximate formula for efficiently estimating leave-one-out cross-validation error in Bayesian linear regression, reducing computational costs for large datasets.

Contribution

It develops a novel analytical approximation for CV error in Bayesian linear regression, avoiding the need for extensive re-computation.

Findings

The formula accurately estimates CV error in synthetic models.

Application to supernova data confirms practical usefulness.

Reduces computational cost significantly for large datasets.

Abstract

Cross-validation (CV) is a technique for evaluating the ability of statistical models/learning systems based on a given data set. Despite its wide applicability, the rather heavy computational cost can prevent its use as the system size grows. To resolve this difficulty in the case of Bayesian linear regression, we develop a formula for evaluating the leave-one-out CV error approximately without actually performing CV. The usefulness of the developed formula is tested by statistical mechanical analysis for a synthetic model. This is confirmed by application to a real-world supernova data set as well.