Disentangled Feature Importance

TL;DR

This paper introduces Disentangled Feature Importance (DFI), a novel nonparametric method that accurately assesses feature contributions by transforming correlated predictors into independent latent features using entropic optimal transport, improving interpretability and efficiency.

Contribution

DFI provides a new nonparametric approach for feature importance that disentangles correlated features via entropic optimal transport, with theoretical guarantees and computational efficiency.

Findings

DFI accurately decomposes importance scores under arbitrary feature dependencies.

DFI achieves root-n consistency and asymptotic normality for importance estimators.

DFI avoids computationally intensive submodel refitting and conditional distribution estimation.

Abstract

Feature importance (FI) measures are widely used to assess the contributions of predictors to an outcome, but they may target different notions of relevance. When predictors are correlated, traditional statistical FI methods are often tailored for feature selection and correlation can therefore be treated as conditional redundancy. By contrast, for model interpretation, FI is more naturally defined through marginal predictive relevance. In this context, we show that most existing approaches target identical population functionals under squared-error loss and exhibit correlation-induced bias. To address this limitation, we introduce Disentangled Feature Importance (DFI), a nonparametric generalization of the classical $R^2$ decomposition via canonical entropic optimal transport (EOT). DFI transforms correlated features into independent latent features using an EOT coupling for general covariate laws, including mixed and discrete settings. Importance scores are computed in this disentangled space and attributed back through the transition kernel's sensitivity. Under arbitrary feature dependencies, DFI provides a principled decomposition of latent importance scores that sum to the total predictive variability for latent additive models and to interaction-weighted functional ANOVA variances more generally. We develop semiparametric theory for DFI. Under the EOT formulation, we establish root-$n$ consistency and asymptotic normality for nondegenerate importance estimators in the latent space and the original feature space. Notably, our estimators achieve second-order estimation error, which vanishes if both regression function and EOT kernel estimation errors are $o_{\mathbb{P}}(n^{-1/4})$. By design, DFI avoids the computational burden of repeated submodel refitting and the challenges of conditional covariate distribution estimation, thereby achieving computational efficiency.