Representer Point Selection for Explaining Deep Neural Networks

TL;DR

This paper introduces a method to explain neural network predictions by identifying key training points, called representer points, which influence the model's output, offering deeper insights and real-time interpretability.

Contribution

The paper presents a novel representer point selection method that decomposes predictions into training point contributions, enhancing interpretability and scalability over existing influence-based explanations.

Findings

Effectively identifies influential training points for specific predictions.

Provides deeper insights into positive and negative training point impacts.

Enables real-time explanation feedback.

Abstract

We propose to explain the predictions of a deep neural network, by pointing to the set of what we call representer points in the training set, for a given test point prediction. Specifically, we show that we can decompose the pre-activation prediction of a neural network into a linear combination of activations of training points, with the weights corresponding to what we call representer values, which thus capture the importance of that training point on the learned parameters of the network. But it provides a deeper understanding of the network than simply training point influence: with positive representer values corresponding to excitatory training points, and negative values corresponding to inhibitory points, which as we show provides considerably more insight. Our method is also much more scalable, allowing for real-time feedback in a manner not feasible with influence functions.