A survey of cross-validation procedures for model selection

TL;DR

This survey reviews various cross-validation procedures for model selection, relating empirical findings to recent theoretical advances and providing guidelines for choosing appropriate methods based on specific problem features.

Contribution

It systematically connects existing empirical results with recent theoretical insights on cross-validation, offering practical guidelines for selecting procedures.

Findings

Different cross-validation methods have varying performance depending on problem features.

The survey clarifies the distinction between empirical and theoretical results in cross-validation.

Guidelines are provided for choosing the most suitable cross-validation procedure.

Abstract

Used to estimate the risk of an estimator or to perform model selection, cross-validation is a widespread strategy because of its simplicity and its apparent universality. Many results exist on the model selection performances of cross-validation procedures. This survey intends to relate these results to the most recent advances of model selection theory, with a particular emphasis on distinguishing empirical statements from rigorous theoretical results. As a conclusion, guidelines are provided for choosing the best cross-validation procedure according to the particular features of the problem in hand.