Bootstrap Bias Corrected Cross Validation applied to Super Learning

TL;DR

This paper introduces Bootstrap Bias Correction, a resampling method for verifying super learner models, which offers a cost-effective alternative to nested cross validation with comparable accuracy.

Contribution

It applies Bootstrap Bias Correction to super learning verification, demonstrating its effectiveness compared to traditional nested cross validation.

Findings

Bootstrap Bias Correction is accurate and cost-efficient.

It performs well across artificial and biomedical datasets.

Resampling method reduces computational cost significantly.

Abstract

Super learner algorithm can be applied to combine results of multiple base learners to improve quality of predictions. The default method for verification of super learner results is by nested cross validation. It has been proposed by Tsamardinos et al., that nested cross validation can be replaced by resampling for tuning hyper-parameters of the learning algorithms. We apply this idea to verification of super learner and compare with other verification methods, including nested cross validation. Tests were performed on artificial data sets of diverse size and on seven real, biomedical data sets. The resampling method, called Bootstrap Bias Correction, proved to be a reasonably precise and very cost-efficient alternative for nested cross validation.