Adaptive MSD-Splitting: Enhancing C4.5 and Random Forests for Skewed Continuous Attributes

TL;DR

This paper introduces Adaptive MSD-Splitting, a dynamic discretization method for decision trees that improves accuracy on skewed data while maintaining computational efficiency, and integrates it into Random Forests.

Contribution

It proposes AMSD, an adaptive discretization technique that adjusts to data skewness, and demonstrates its effectiveness within ensemble learning frameworks.

Findings

AMSD improves accuracy by 2-4% over standard MSD-Splitting.

RF-AMSD achieves state-of-the-art accuracy with reduced computational costs.

The method maintains near-linear time complexity despite adaptive adjustments.

Abstract

The discretization of continuous numerical attributes remains a persistent computational bottleneck in the induction of decision trees, particularly as dataset dimensions scale. Building upon the recently proposed MSD-Splitting technique -- which bins continuous data using the empirical mean and standard deviation to dramatically improve the efficiency and accuracy of the C4.5 algorithm -- we introduce Adaptive MSD-Splitting (AMSD). While standard MSD-Splitting is highly effective for approximately symmetric distributions, its rigid adherence to fixed one-standard-deviation cutoffs can lead to catastrophic information loss in highly skewed data, a common artifact in real-world biomedical and financial datasets. AMSD addresses this by dynamically adjusting the standard deviation multiplier based on feature skewness, narrowing intervals in dense regions to preserve discriminative resolution. Furthermore, we integrate AMSD into ensemble methods, specifically presenting the Random Forest-AMSD (RF-AMSD) framework. Empirical evaluations on the Census Income, Heart Disease, Breast Cancer, and Forest Covertype datasets demonstrate that AMSD yields a 2-4% accuracy improvement over standard MSD-Splitting, while maintaining near-identical O(N) time complexity reductions compared to the O(N log N) exhaustive search. Our Random Forest extension achieves state-of-the-art accuracy at a fraction of standard computational costs, confirming the viability of adaptive statistical binning in large-scale ensemble learning architectures.