Repeated undersampling in PrInDT (RePrInDT): Variation in undersampling and prediction, and ranking of predictors in ensembles

TL;DR

This paper extends the PrInDT method to include various undersampling strategies, predictor stratification, and threshold adjustments, demonstrating their effects on model accuracy and predictor importance in ensemble learning.

Contribution

It introduces new undersampling techniques, evaluates predictor stratification, and proposes a predictor importance ranking method within ensemble models.

Findings

Careful selection of undersampling percentages improves balanced accuracy.

Predictor stratification does not significantly enhance accuracy.

Lowering the prediction threshold can substitute undersampling for better class prediction.

Abstract

In this paper, we extend our PrInDT method (Weihs & Buschfeld 2021a) towards undersampling with different percentages of the smaller and the larger classes (psmall and plarge), stratification of predictors, varying the prediction threshold, and measuring variable importance in ensembles. An application of these methods to a linguistic example suggests the following: 1. In undersampling, a careful selection of the percentages plarge and psmall is important for building models with high balanced accuracies; 2. Stratification of predictors does not majorly enhance balanced accuracies; 3. Lowering the prediction threshold for the smaller class turns out to be an alternative method to undersampling because it increases the likelihood of the smaller class being selected. Finally, we introduce a method for ranking predictor importance that allows for a straightforward interpretation of the results.