CLIMB: Class-imbalanced Learning Benchmark on Tabular Data

TL;DR

CLIMB is a comprehensive benchmark for class-imbalanced learning on tabular data, providing datasets, algorithms, and insights to improve method evaluation and development.

Contribution

The paper introduces CLIMB, a unified benchmark with datasets and implementations for evaluating CIL algorithms on tabular data, facilitating fair comparison and analysis.

Findings

Naive rebalancing methods have limitations.

Ensemble methods show improved performance.

Data quality significantly impacts results.

Abstract

Class-imbalanced learning (CIL) on tabular data is important in many real-world applications where the minority class holds the critical but rare outcomes. In this paper, we present CLIMB, a comprehensive benchmark for class-imbalanced learning on tabular data. CLIMB includes 73 real-world datasets across diverse domains and imbalance levels, along with unified implementations of 29 representative CIL algorithms. Built on a high-quality open-source Python package with unified API designs, detailed documentation, and rigorous code quality controls, CLIMB supports easy implementation and comparison between different CIL algorithms. Through extensive experiments, we provide practical insights on method accuracy and efficiency, highlighting the limitations of naive rebalancing, the effectiveness of ensembles, and the importance of data quality. Our code, documentation, and examples are available at https://github.com/ZhiningLiu1998/imbalanced-ensemble.