Explaining Deep Neural Networks by Leveraging Intrinsic Methods

TL;DR

This paper advances explainable AI by developing methods to interpret deep neural networks through self-explanatory designs, neuron analysis, and evaluating explanatory techniques in visual analytics.

Contribution

It introduces new interpretability techniques including self-explanatory architectures, neuron analysis, and assessment of explanation methods in visual analytics.

Findings

Proposed self-explanatory neural network designs with external memory and prototype layers

Uncovered new insights into neuron activation phenomena

Evaluated the effectiveness of explanation techniques in visual analytics systems

Abstract

Despite their impact on the society, deep neural networks are often regarded as black-box models due to their intricate structures and the absence of explanations for their decisions. This opacity poses a significant challenge to AI systems wider adoption and trustworthiness. This thesis addresses this issue by contributing to the field of eXplainable AI, focusing on enhancing the interpretability of deep neural networks. The core contributions lie in introducing novel techniques aimed at making these networks more interpretable by leveraging an analysis of their inner workings. Specifically, the contributions are threefold. Firstly, the thesis introduces designs for self-explanatory deep neural networks, such as the integration of external memory for interpretability purposes and the usage of prototype and constraint-based layers across several domains. Secondly, this research delves into novel investigations on neurons within trained deep neural networks, shedding light on overlooked phenomena related to their activation values. Lastly, the thesis conducts an analysis of the application of explanatory techniques in the field of visual analytics, exploring the maturity of their adoption and the potential of these systems to convey explanations to users effectively.