Learning Important Features Through Propagating Activation Differences

TL;DR

DeepLIFT is a novel method for interpreting neural network predictions by backpropagating contribution scores based on differences from reference activations, offering advantages over gradient-based methods.

Contribution

It introduces DeepLIFT, a new approach for decomposing neural network outputs into feature contributions using activation differences, improving interpretability.

Findings

DeepLIFT provides more accurate feature importance scores than gradient methods.

It can efficiently compute contributions in a single backward pass.

Demonstrated effectiveness on MNIST and genomic data.

Abstract

The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, ICML slides: bit.ly/deeplifticmlslides, ICML talk: https://vimeo.com/238275076, code: http://goo.gl/RM8jvH.