Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples

TL;DR

This paper introduces a new approach to enhance the interpretability of deep neural networks by reducing neuron ambiguity through adversarial training and a novel consistency metric.

Contribution

It proposes a quantitative metric for neuron consistency, reveals neuron ambiguity via adversarial examples, and develops an adversarial training method to improve interpretability.

Findings

Neuron ambiguity can be quantified with a new metric.

Adversarial examples expose neuron ambiguity.

Adversarial training with a consistency loss improves interpretability.

Abstract

Sometimes it is not enough for a DNN to produce an outcome. For example, in applications such as healthcare, users need to understand the rationale of the decisions. Therefore, it is imperative to develop algorithms to learn models with good interpretability (Doshi-Velez 2017). An important factor that leads to the lack of interpretability of DNNs is the ambiguity of neurons, where a neuron may fire for various unrelated concepts. This work aims to increase the interpretability of DNNs on the whole image space by reducing the ambiguity of neurons. In this paper, we make the following contributions: 1) We propose a metric to evaluate the consistency level of neurons in a network quantitatively. 2) We find that the learned features of neurons are ambiguous by leveraging adversarial examples. 3) We propose to improve the consistency of neurons on adversarial example subset by an adversarial training algorithm with a consistent loss.