A redescription mining framework for post-hoc explaining and relating deep learning models

TL;DR

This paper introduces a novel, architecture-independent framework for post-hoc explanation of deep learning models using redescriptions, enabling detailed cohort analysis and interpretability of neuron activations across various models and complex labels.

Contribution

The framework is the first to use redescriptions for explaining and relating deep learning models, supporting complex labels and offering both pedagogical and decompositional rule extraction methods.

Findings

Enables cohort analysis of DLMs through statistically significant redescriptions.

Supports coupling neurons with target labels or descriptive attributes.

Works independently of neural network architecture.

Abstract

Deep learning models (DLMs) achieve increasingly high performance both on structured and unstructured data. They significantly extended applicability of machine learning to various domains. Their success in making predictions, detecting patterns and generating new data made significant impact on science and industry. Despite these accomplishments, DLMs are difficult to explain because of their enormous size. In this work, we propose a novel framework for post-hoc explaining and relating DLMs using redescriptions. The framework allows cohort analysis of arbitrary DLMs by identifying statistically significant redescriptions of neuron activations. It allows coupling neurons to a set of target labels or sets of descriptive attributes, relating layers within a single DLM or associating different DLMs. The proposed framework is independent of the artificial neural network architecture and can work with more complex target labels (e.g. multi-label or multi-target scenario). Additionally, it can emulate both pedagogical and decompositional approach to rule extraction. The aforementioned properties of the proposed framework can increase explainability and interpretability of arbitrary DLMs by providing different information compared to existing explainable-AI approaches.