The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models

TL;DR

This paper introduces a faster, more versatile influence estimation method for machine learning models using information geometry, improving computational efficiency and applicability in non-convex settings.

Contribution

It presents a novel influence function estimation algorithm based on information geometry, enhancing speed and versatility over existing Newton step-based methods.

Findings

The new method is computationally faster than traditional influence estimation techniques.

It remains effective in non-convex model scenarios.

Demonstrates versatility across various applications.

Abstract

Quantifying the influence of infinitesimal changes in training data on model performance is crucial for understanding and improving machine learning models. In this work, we reformulate this problem as a weighted empirical risk minimization and enhance existing influence function-based methods by using information geometry to derive a new algorithm to estimate influence. Our formulation proves versatile across various applications, and we further demonstrate in simulations how it remains informative even in non-convex cases. Furthermore, we show that our method offers significant computational advantages over current Newton step-based methods.