Targeted Learning for Variable Importance

TL;DR

This paper introduces a targeted learning framework to improve the robustness and finite-sample accuracy of variable importance inference in machine learning, especially for conditional permutation importance.

Contribution

We develop a novel targeted learning method that enhances finite-sample robustness for variable importance metrics without sacrificing asymptotic efficiency.

Findings

Retains asymptotic efficiency of traditional methods

Maintains similar computational complexity

Provides improved accuracy in finite samples

Abstract

Variable importance is one of the most widely used measures for interpreting machine learning with significant interest from both statistics and machine learning communities. Recently, increasing attention has been directed toward uncertainty quantification in these metrics. Current approaches largely rely on one-step procedures, which, while asymptotically efficient, can present higher sensitivity and instability in finite sample settings. To address these limitations, we propose a novel method by employing the targeted learning (TL) framework, designed to enhance robustness in inference for variable importance metrics. Our approach is particularly suited for conditional permutation variable importance. We show that it (i) retains the asymptotic efficiency of traditional methods, (ii) maintains comparable computational complexity, and (iii) delivers improved accuracy, especially in finite sample contexts. We further support these findings with numerical experiments that illustrate the practical advantages of our method and validate the theoretical results.