Statistical and Computational Guarantees for Influence Diagnostics

TL;DR

This paper provides finite-sample statistical and computational guarantees for influence diagnostics like influence functions and perturbations, demonstrating their effectiveness in models including generalized linear and large attention-based models.

Contribution

It establishes the first finite-sample statistical and computational bounds for influence diagnostics with efficient inverse-Hessian-vector product methods.

Findings

Finite-sample statistical bounds for influence diagnostics.

Computational complexity bounds for influence functions.

Empirical validation on generalized linear and attention-based models.

Abstract

Influence diagnostics such as influence functions and approximate maximum influence perturbations are popular in machine learning and in AI domain applications. Influence diagnostics are powerful statistical tools to identify influential datapoints or subsets of datapoints. We establish finite-sample statistical bounds, as well as computational complexity bounds, for influence functions and approximate maximum influence perturbations using efficient inverse-Hessian-vector product implementations. We illustrate our results with generalized linear models and large attention based models on synthetic and real data.