Semantic Reasoning from Model-Agnostic Explanations

TL;DR

ReEx is a novel method that enhances black-box model explanations by translating feature-level explanations into semantically meaningful descriptions using ontologies, improving interpretability especially in biological contexts.

Contribution

ReEx introduces a way to generalize and semantically interpret explanations from tools like SHAP using ontologies, bridging the gap between raw explanations and human-understandable knowledge.

Findings

ReEx produces more informative, compact explanations.

Semantic explanations outperform direct ontology mappings.

Applicable to biological datasets with improved interpretability.

Abstract

With the wide adoption of black-box models, instance-based \emph{post hoc} explanation tools, such as LIME and SHAP became increasingly popular. These tools produce explanations, pinpointing contributions of key features associated with a given prediction. However, the obtained explanations remain at the raw feature level and are not necessarily understandable by a human expert without extensive domain knowledge. We propose ReEx (Reasoning with Explanations), a method applicable to explanations generated by arbitrary instance-level explainers, such as SHAP. By using background knowledge in the form of ontologies, ReEx generalizes instance explanations in a least general generalization-like manner. The resulting symbolic descriptions are specific for individual classes and offer generalizations based on the explainer's output. The derived semantic explanations are potentially more informative, as they describe the key attributes in the context of more general background knowledge, e.g., at the biological process level. We showcase ReEx's performance on nine biological data sets, showing that compact, semantic explanations can be obtained and are more informative than generic ontology mappings that link terms directly to feature names. ReEx is offered as a simple-to-use Python library and is compatible with tools such as SHAP and similar. To our knowledge, this is one of the first methods that directly couples semantic reasoning with contemporary model explanation methods. This paper is a preprint. Full version's doi is: 10.1109/SAMI50585.2021.9378668