FLEX: Feature Importance from Layered Counterfactual Explanations

TL;DR

FLEX is a versatile framework that transforms counterfactual explanations into feature importance scores across different levels, enhancing interpretability and actionable insights in high-stakes machine learning applications.

Contribution

FLEX introduces a novel, model- and domain-agnostic method to derive feature importance from counterfactuals, capturing both local and global feature influence in a unified framework.

Findings

FLEX's global rankings align with SHAP and reveal additional feature drivers.

Regional analysis uncovers context-specific factors missed by global summaries.

FLEX improves interpretability and supports intervention in risk-sensitive tasks.

Abstract

Machine learning models achieve state-of-the-art performance across domains, yet their lack of interpretability limits safe deployment in high-stakes settings. Counterfactual explanations are widely used to provide actionable "what-if" recourse, but they typically remain instance-specific and do not quantify which features systematically drive outcome changes within coherent regions of the feature space or across an entire dataset. We introduce FLEX (Feature importance from Layered counterfactual EXplanations), a model- and domain-agnostic framework that converts sets of counterfactuals into feature change frequency scores at local, regional, and global levels. FLEX generalises local change-frequency measures by aggregating across instances and neighbourhoods, offering interpretable rankings that reflect how often each feature must change to flip predictions. The framework is compatible with different counterfactual generation methods, allowing users to emphasise characteristics such as sparsity, feasibility, or actionability, thereby tailoring the derived feature importances to practical constraints. We evaluate FLEX on two contrasting tabular tasks: traffic accident severity prediction and loan approval, and compare FLEX to SHAP- and LIME-derived feature importance values. Results show that (i) FLEX's global rankings correlate with SHAP while surfacing additional drivers, and (ii) regional analyses reveal context-specific factors that global summaries miss. FLEX thus bridges the gap between local recourse and global attribution, supporting transparent and intervention-oriented decision-making in risk-sensitive applications.