Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Features

TL;DR

This paper introduces Symbolic XAI, a framework that explains AI models through human-understandable logical relationships between features, capturing abstract reasoning in diverse domains.

Contribution

It proposes a novel symbolic explanation framework that utilizes logical relationships and higher-order relevance methods to enhance interpretability of AI models.

Findings

Effective in NLP, vision, and quantum chemistry domains

Provides human-readable logical explanations

Captures abstract reasoning behind model predictions

Abstract

Explainable Artificial Intelligence (XAI) plays a crucial role in fostering transparency and trust in AI systems, where traditional XAI approaches typically offer one level of abstraction for explanations, often in the form of heatmaps highlighting single or multiple input features. However, we ask whether abstract reasoning or problem-solving strategies of a model may also be relevant, as these align more closely with how humans approach solutions to problems. We propose a framework, called Symbolic XAI, that attributes relevance to symbolic queries expressing logical relationships between input features, thereby capturing the abstract reasoning behind a model's predictions. The methodology is built upon a simple yet general multi-order decomposition of model predictions. This decomposition can be specified using higher-order propagation-based relevance methods, such as GNN-LRP, or perturbation-based explanation methods commonly used in XAI. The effectiveness of our framework is demonstrated in the domains of natural language processing (NLP), vision, and quantum chemistry (QC), where abstract symbolic domain knowledge is abundant and of significant interest to users. The Symbolic XAI framework provides an understanding of the model's decision-making process that is both flexible for customization by the user and human-readable through logical formulas.