# Pruning variable selection ensembles

## TL;DR

This paper introduces a novel ordering-based selective ensemble learning method for variable selection, which prunes ensembles to improve accuracy and reduce false discoveries, demonstrated through experiments with stability selection.

## Contribution

The paper proposes a new greedy sorting and early stopping strategy for ensemble pruning in variable selection, enhancing accuracy and reducing false positives.

## Key findings

- Pruned ensembles outperform full ensembles in accuracy.
- The method reduces false discovery rates.
- Experimental results validate the effectiveness of the approach.

## Abstract

In the context of variable selection, ensemble learning has gained increasing interest due to its great potential to improve selection accuracy and to reduce false discovery rate. A novel ordering-based selective ensemble learning strategy is designed in this paper to obtain smaller but more accurate ensembles. In particular, a greedy sorting strategy is proposed to rearrange the order by which the members are included into the integration process. Through stopping the fusion process early, a smaller subensemble with higher selection accuracy can be obtained. More importantly, the sequential inclusion criterion reveals the fundamental strength-diversity trade-off among ensemble members. By taking stability selection (abbreviated as StabSel) as an example, some experiments are conducted with both simulated and real-world data to examine the performance of the novel algorithm. Experimental results demonstrate that pruned StabSel generally achieves higher selection accuracy and lower false discovery rates than StabSel and several other benchmark methods.