Interpreting the Predictions of Complex ML Models by Layer-wise Relevance Propagation

TL;DR

This paper discusses a technique called Layer-wise Relevance Propagation that helps interpret complex deep neural network predictions by decomposing their decisions into input variable contributions, enhancing transparency.

Contribution

It summarizes a recent method for explaining DNN predictions through input variable decomposition, improving interpretability of complex models.

Findings

Enables understanding of DNN decision-making process

Provides input variable relevance scores

Improves transparency in sensitive applications

Abstract

Complex nonlinear models such as deep neural network (DNNs) have become an important tool for image classification, speech recognition, natural language processing, and many other fields of application. These models however lack transparency due to their complex nonlinear structure and to the complex data distributions to which they typically apply. As a result, it is difficult to fully characterize what makes these models reach a particular decision for a given input. This lack of transparency can be a drawback, especially in the context of sensitive applications such as medical analysis or security. In this short paper, we summarize a recent technique introduced by Bach et al. [1] that explains predictions by decomposing the classification decision of DNN models in terms of input variables.