autoBagging: Learning to Rank Bagging Workflows with Metalearning

TL;DR

autoBagging is an autoML system that uses metalearning and ranking techniques to select optimal bagging workflows based on dataset metadata, outperforming average ranking and approaching ideal model performance.

Contribution

It introduces a novel autoML approach that leverages metalearning and learning to rank for workflow selection, differing from traditional optimization-based methods.

Findings

autoBagging outperforms the Average Rank method.

autoBagging achieves results statistically close to the ideal model.

The system is publicly available as an R package on CRAN.

Abstract

Machine Learning (ML) has been successfully applied to a wide range of domains and applications. One of the techniques behind most of these successful applications is Ensemble Learning (EL), the field of ML that gave birth to methods such as Random Forests or Boosting. The complexity of applying these techniques together with the market scarcity on ML experts, has created the need for systems that enable a fast and easy drop-in replacement for ML libraries. Automated machine learning (autoML) is the field of ML that attempts to answers these needs. Typically, these systems rely on optimization techniques such as bayesian optimization to lead the search for the best model. Our approach differs from these systems by making use of the most recent advances on metalearning and a learning to rank approach to learn from metadata. We propose autoBagging, an autoML system that automatically ranks 63 bagging workflows by exploiting past performance and dataset characterization. Results on 140 classification datasets from the OpenML platform show that autoBagging can yield better performance than the Average Rank method and achieve results that are not statistically different from an ideal model that systematically selects the best workflow for each dataset. For the purpose of reproducibility and generalizability, autoBagging is publicly available as an R package on CRAN.