Selecting Classifiers by Pooling over Cross-Validation Results in More Consistency in Small-Sample Classification of Atrial Flutter Localization

TL;DR

This paper proposes a modified cross-validation method for selecting classifiers that improves consistency and reduces bias and variance, demonstrated on atrial flutter localization in clinical AI applications.

Contribution

It introduces a new classifier selection approach based on pooling over cross-validation results, enhancing stability and reducing bias in small-sample medical classification tasks.

Findings

More consistent classifiers with similar average performance

Reduced extra-sample loss variance

Lower feature bias in classifier selection

Abstract

Selecting learning machines such as classifiers is an important task when using AI in the clinic. K-fold crossvalidation is a practical technique that allows simple inference of such machines. However, the recipe generates many models and does not provide a means to determine the best one. In this paper, a modified recipe is presented, that generates more consistent machines with similar on-average performance, but less extra-sample loss variance and less feature bias. A use case is provided by applying the recipe onto the atrial flutter localization problem.