Space Explanations of Neural Network Classification

TL;DR

This paper introduces Space Explanations, a logic-based method providing provable guarantees for neural network behavior over input regions, improving interpretability with automated, meaningful explanations.

Contribution

It proposes a novel logic-based framework using Craig interpolation and unsatisfiable core techniques to generate provable, region-based explanations for neural networks.

Findings

More meaningful explanations than state-of-the-art methods

Effective across small to large neural network models

Provides provable guarantees of network behavior

Abstract

We present a novel logic-based concept called Space Explanations for classifying neural networks that gives provable guarantees of the behavior of the network in continuous areas of the input feature space. To automatically generate space explanations, we leverage a range of flexible Craig interpolation algorithms and unsatisfiable core generation. Based on real-life case studies, ranging from small to medium to large size, we demonstrate that the generated explanations are more meaningful than those computed by state-of-the-art.