BPE: Behavioral Profiling Ensemble

TL;DR

The paper introduces the Behavioral Profiling Ensemble (BPE), a novel model-centric ensemble method that improves predictive accuracy and efficiency by using intrinsic behavioral profiles of models for dynamic aggregation.

Contribution

BPE presents a new approach to ensemble learning by constructing behavioral profiles for models and using them for dynamic, model-centric aggregation, outperforming existing methods.

Findings

BPE outperforms state-of-the-art DES methods in accuracy.

BPE reduces computational and storage costs.

BPE demonstrates effectiveness on 42 real-world datasets.

Abstract

In the field of machine learning, ensemble learning is widely recognized as a pivotal strategy for pushing the boundaries of predictive performance. Traditional static ensemble methods typically assign weights by treating each base learner as a whole, thereby overlooking that individual models exhibit varying competence across different regions of the instance space. Dynamic Ensemble Selection (DES) was introduced to address this limitation. However, both static and dynamic approaches predominantly rely on inter-model differences as the basis for integration; this inter-model perspective neglects models' intrinsic characteristics and often requires heavy reliance on reference sets for competence estimation. We propose the Behavioral Profiling Ensemble (BPE) framework, which introduces a model-centric integration paradigm. Unlike traditional methods, BPE constructs an intrinsic behavioral profile $\mathcal{P}_k$ for each model and derives aggregation weights from the deviation between a model's response to a test instance and its established profile; in this work, we instantiate $\mathcal{P}_k$ with entropy-based summary statistics (e.g., mean and variance). Extensive experiments on 42 real-world datasets show that BPE-derived algorithms outperform state-of-the-art DES baselines, increasing predictive accuracy while reducing computational and storage overhead.