Measuring and improving the quality of visual explanations

TL;DR

This paper introduces a new evaluation procedure for visual explanations of neural networks, assesses various explanation sources, and examines the impact of bias parameters on explanation quality and classifier performance.

Contribution

It proposes a novel method for evaluating visual explanations and analyzes the effects of combining sources and bias parameters on explanation effectiveness.

Findings

Combining multiple explanation sources improves interpretability.

Bias parameters significantly influence explanation quality.

Assessment of explanations reveals limitations of current methods.

Abstract

The ability of to explain neural network decisions goes hand in hand with their safe deployment. Several methods have been proposed to highlight features important for a given network decision. However, there is no consensus on how to measure effectiveness of these methods. We propose a new procedure for evaluating explanations. We use it to investigate visual explanations extracted from a range of possible sources in a neural network. We quantify the benefit of combining these sources and challenge a recent appeal for taking bias parameters into account. We support our conclusions with a general assessment of the impact of bias parameters in ImageNet classifiers