Decoupling Deep Learning for Interpretable Image Recognition

TL;DR

This paper introduces DProtoNet, a decoupled prototype-based neural network that enhances interpretability without sacrificing accuracy, using novel modules for feature generation, prototype learning, and explanation.

Contribution

The paper proposes a decoupled architecture for prototype-based networks, improving interpretability and accuracy simultaneously through new modules and learning strategies.

Findings

Achieved 5% accuracy improvement over previous methods

Demonstrated state-of-the-art interpretability on multiple datasets

Validated effectiveness on both general and medical image datasets

Abstract

The interpretability of neural networks has recently received extensive attention. Previous prototype-based explainable networks involved prototype activation in both reasoning and interpretation processes, requiring specific explainable structures for the prototype, thus making the network less accurate as it gains interpretability. Therefore, the decoupling prototypical network (DProtoNet) was proposed to avoid this problem. This new model contains encoder, inference, and interpretation modules. As regards the encoder module, unrestricted feature masks were presented to generate expressive features and prototypes. Regarding the inference module, a multi-image prototype learning method was introduced to update prototypes so that the network can learn generalized prototypes. Finally, concerning the interpretation module, a multiple dynamic masks (MDM) decoder was suggested to explain the neural network, which generates heatmaps using the consistent activation of the original image and mask image at the detection nodes of the network. It decouples the inference and interpretation modules of a prototype-based network by avoiding the use of prototype activation to explain the network's decisions in order to simultaneously improve the accuracy and interpretability of the neural network. The multiple public general and medical datasets were tested, and the results confirmed that our method could achieve a 5% improvement in accuracy and state-of-the-art interpretability compared with previous methods.