GAN-based Generation and Automatic Selection of Explanations for Neural Networks

TL;DR

This paper introduces a GAN-based method and a Fréchet Inception Distance metric to efficiently generate and select interpretable explanations for neural network activations, reducing manual evaluation.

Contribution

It proposes a novel GAN-based explanation generation technique combined with an FID-based metric for hyper-parameter selection, streamlining interpretability of neural networks.

Findings

FID metric effectively selects hyper-parameters for interpretable examples

Generated explanations align with vocal and non-vocal features

Approach reduces manual effort in explanation generation

Abstract

One way to interpret trained deep neural networks (DNNs) is by inspecting characteristics that neurons in the model respond to, such as by iteratively optimising the model input (e.g., an image) to maximally activate specific neurons. However, this requires a careful selection of hyper-parameters to generate interpretable examples for each neuron of interest, and current methods rely on a manual, qualitative evaluation of each setting, which is prohibitively slow. We introduce a new metric that uses Fr\'echet Inception Distance (FID) to encourage similarity between model activations for real and generated data. This provides an efficient way to evaluate a set of generated examples for each setting of hyper-parameters. We also propose a novel GAN-based method for generating explanations that enables an efficient search through the input space and imposes a strong prior favouring realistic outputs. We apply our approach to a classification model trained to predict whether a music audio recording contains singing voice. Our results suggest that this proposed metric successfully selects hyper-parameters leading to interpretable examples, avoiding the need for manual evaluation. Moreover, we see that examples synthesised to maximise or minimise the predicted probability of singing voice presence exhibit vocal or non-vocal characteristics, respectively, suggesting that our approach is able to generate suitable explanations for understanding concepts learned by a neural network.