Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers

TL;DR

This paper extends layer-wise relevance propagation to neural networks with local renormalization layers, enabling interpretability of models with common non-linearities across popular datasets.

Contribution

It introduces a novel method to apply relevance propagation to networks with local renormalization layers, which was previously not supported.

Findings

Effective relevance propagation on local renormalization layers

Improved interpretability of CNNs with renormalization layers

Validated on CIFAR-10, Imagenet, and MIT Places datasets

Abstract

Layer-wise relevance propagation is a framework which allows to decompose the prediction of a deep neural network computed over a sample, e.g. an image, down to relevance scores for the single input dimensions of the sample such as subpixels of an image. While this approach can be applied directly to generalized linear mappings, product type non-linearities are not covered. This paper proposes an approach to extend layer-wise relevance propagation to neural networks with local renormalization layers, which is a very common product-type non-linearity in convolutional neural networks. We evaluate the proposed method for local renormalization layers on the CIFAR-10, Imagenet and MIT Places datasets.