Novel split quality measures for stratified multilabel Cross Validation with application to large and sparse gene ontology datasets

TL;DR

This paper introduces new split quality measures and an algorithm called optisplit for creating better cross validation splits in multilabel datasets, especially addressing class imbalance in large, sparse gene ontology data.

Contribution

The paper proposes improved split quality measures and an efficient algorithm, optisplit, tailored for multilabel data with class imbalance, enhancing cross validation in large datasets.

Findings

Optisplit produces more balanced cross validation splits.

Optisplit is computationally efficient for large gene ontology datasets.

Existing measures do not adequately handle multilabel class imbalance.

Abstract

Multilabel learning is an important topic in machine learning research. Evaluating models in multilabel settings requires specific cross validation methods designed for multilabel data. In this article, we show that the most widely used cross validation split quality measures do not behave adequately with multilabel data that has strong class imbalance. We present improved measures and an algorithm, optisplit, for optimising cross validations splits. We present an extensive comparison of various types of cross validation methods in which we show that optisplit produces more even cross validation splits than the existing methods and that it is fast enough to be used on big Gene Ontology (GO) datasets.