Newfluence: Boosting Model interpretability and Understanding in High Dimensions

TL;DR

This paper investigates the limitations of influence functions in high-dimensional AI models and introduces Newfluence, a new method that improves interpretability and diagnostic accuracy in such settings.

Contribution

The paper identifies the shortcomings of influence functions in high dimensions and proposes Newfluence, an alternative approximation with better accuracy and similar efficiency.

Findings

Influence functions are unreliable in high-dimensional regimes.

Newfluence significantly improves interpretability accuracy.

The high-dimensional framework applies to other interpretability techniques.

Abstract

The increasing complexity of machine learning (ML) and artificial intelligence (AI) models has created a pressing need for tools that help scientists, engineers, and policymakers interpret and refine model decisions and predictions. Influence functions, originating from robust statistics, have emerged as a popular approach for this purpose. However, the heuristic foundations of influence functions rely on low-dimensional assumptions where the number of parameters $p$ is much smaller than the number of observations $n$. In contrast, modern AI models often operate in high-dimensional regimes with large $p$, challenging these assumptions. In this paper, we examine the accuracy of influence functions in high-dimensional settings. Our theoretical and empirical analyses reveal that influence functions cannot reliably fulfill their intended purpose. We then introduce an alternative approximation, called Newfluence, that maintains similar computational efficiency while offering significantly improved accuracy. Newfluence is expected to provide more accurate insights than many existing methods for interpreting complex AI models and diagnosing their issues. Moreover, the high-dimensional framework we develop in this paper can also be applied to analyze other popular techniques, such as Shapley values.