ReasonX: Declarative Reasoning on Explanations

TL;DR

ReasonX introduces a declarative, interactive explanation tool for decision trees that integrates symbolic knowledge and reasoning at multiple abstraction levels using linear constraints and MILP, enhancing XAI capabilities.

Contribution

It presents a novel reasoning-based XAI framework combining declarative queries, symbolic background knowledge, and MILP reasoning over decision tree models.

Findings

Effective reasoning over features with MILP

Supports background knowledge integration

Outperforms existing XAI tools in experiments

Abstract

Explaining opaque Machine Learning (ML) models has become an increasingly important challenge. However, current eXplanation in AI (XAI) methods suffer several shortcomings, including insufficient abstraction, limited user interactivity, and inadequate integration of symbolic knowledge. We propose ReasonX, an explanation tool based on expressions (or, queries) in a closed algebra of operators over theories of linear constraints. ReasonX provides declarative and interactive explanations for decision trees, which may represent the ML models under analysis or serve as global or local surrogate models for any black-box predictor. Users can express background or common sense knowledge as linear constraints. This allows for reasoning at multiple levels of abstraction, ranging from fully specified examples to under-specified or partially constrained ones. ReasonX leverages Mixed-Integer Linear Programming (MILP) to reason over the features of factual and contrastive instances. We present here the architecture of ReasonX, which consists of a Python layer, closer to the user, and a Constraint Logic Programming (CLP) layer, which implements a meta-interpreter of the query algebra. The capabilities of ReasonX are demonstrated through qualitative examples, and compared to other XAI tools through quantitative experiments.