XStacking: Explanation-Guided Stacked Ensemble Learning

TL;DR

XStacking is an explainable ensemble learning framework that combines dynamic feature transformation with model-agnostic explanations, maintaining high accuracy and improving interpretability across diverse datasets.

Contribution

It introduces a novel, inherently explainable stacking framework that integrates feature transformation and Shapley explanations, addressing interpretability issues in ensemble models.

Findings

Achieves improved predictive performance on 29 datasets.

Enhances interpretability of stacked models.

Maintains accuracy while providing explanations.

Abstract

Ensemble Machine Learning (EML) techniques, especially stacking, have been shown to improve predictive performance by combining multiple base models. However, they are often criticized for their lack of interpretability. In this paper, we introduce XStacking, an effective and inherently explainable framework that addresses this limitation by integrating dynamic feature transformation with model-agnostic Shapley additive explanations. This enables stacked models to retain their predictive accuracy while becoming inherently explainable. We demonstrate the effectiveness of the framework on 29 datasets, achieving improvements in both the predictive effectiveness of the learning space and the interpretability of the resulting models. XStacking offers a practical and scalable solution for responsible ML.