Expressive Explanations of DNNs by Combining Concept Analysis with ILP

TL;DR

This paper presents a method to generate expressive, symbolic explanations for deep neural networks in computer vision by combining concept analysis with inductive logic programming, enhancing transparency and interpretability.

Contribution

It introduces a novel approach that uses inherent network features and concept analysis to produce first-order rule explanations for DNNs, improving interpretability in vision tasks.

Findings

Explanations are faithful to the original model.

Semantic features are effectively mined from network activations.

First-order rules provide more expressive explanations.

Abstract

Explainable AI has emerged to be a key component for black-box machine learning approaches in domains with a high demand for reliability or transparency. Examples are medical assistant systems, and applications concerned with the General Data Protection Regulation of the European Union, which features transparency as a cornerstone. Such demands require the ability to audit the rationale behind a classifier's decision. While visualizations are the de facto standard of explanations, they come short in terms of expressiveness in many ways: They cannot distinguish between different attribute manifestations of visual features (e.g. eye open vs. closed), and they cannot accurately describe the influence of absence of, and relations between features. An alternative would be more expressive symbolic surrogate models. However, these require symbolic inputs, which are not readily available in most computer vision tasks. In this paper we investigate how to overcome this: We use inherent features learned by the network to build a global, expressive, verbal explanation of the rationale of a feed-forward convolutional deep neural network (DNN). The semantics of the features are mined by a concept analysis approach trained on a set of human understandable visual concepts. The explanation is found by an Inductive Logic Programming (ILP) method and presented as first-order rules. We show that our explanation is faithful to the original black-box model. The code for our experiments is available at https://github.com/mc-lovin-mlem/concept-embeddings-and-ilp/tree/ki2020.