# The exact form of the 'Ockham factor' in model selection

## TL;DR

This paper investigates the 'Ockham factor' in model selection, introducing a new measure called 'flexibility' that unifies Bayesian and Frequentist approaches by decomposing log-evidence into fit and complexity.

## Contribution

It introduces 'flexibility' as a new model complexity measure, providing an exact decomposition of log-evidence applicable to both Bayesian and Frequentist frameworks.

## Key findings

- Flexibility equals BIC penalty asymptotically in Gaussian linear models.
- Maximizing evidence aligns with Bayesian and Frequentist principles.
- Cautions against substituting BIC for flexibility in model selection.

## Abstract

We explore the arguments for maximizing the `evidence' as an algorithm for model selection. We show, using a new definition of model complexity which we term `flexibility', that maximizing the evidence should appeal to both Bayesian and Frequentist statisticians. This is due to flexibility's unique position in the exact decomposition of log-evidence into log-fit minus flexibility. In the Gaussian linear model, flexibility is asymptotically equal to the Bayesian Information Criterion (BIC) penalty, but we caution against using BIC in place of flexibility for model selection.