Integrated Influence: Data Attribution with Baseline

TL;DR

This paper introduces Integrated Influence, a new data attribution method that incorporates a baseline to better quantify training sample influence, addressing limitations of existing leave-one-out approaches and providing more reliable explanations.

Contribution

The paper proposes a novel data attribution method with a baseline, unifying influence functions and improving explanation reliability in machine learning models.

Findings

Integrated Influence outperforms existing methods in attribution accuracy.

The method effectively identifies mislabeled training examples.

Influence functions are shown as special cases of the proposed approach.

Abstract

As an effective approach to quantify how training samples influence test sample, data attribution is crucial for understanding data and model and further enhance the transparency of machine learning models. We find that prevailing data attribution methods based on leave-one-out (LOO) strategy suffer from the local-based explanation, as these LOO-based methods only perturb a single training sample, and overlook the collective influence in the training set. On the other hand, the lack of baseline in many data attribution methods reduces the flexibility of the explanation, e.g., failing to provide counterfactual explanations. In this paper, we propose Integrated Influence, a novel data attribution method that incorporates a baseline approach. Our method defines a baseline dataset, follows a data degeneration process to transition the current dataset to the baseline, and accumulates the influence of each sample throughout this process. We provide a solid theoretical framework for our method, and further demonstrate that popular methods, such as influence functions, can be viewed as special cases of our approach. Experimental results show that Integrated Influence generates more reliable data attributions compared to existing methods in both data attribution task and mislablled example identification task.