Explaining Deep Tractable Probabilistic Models: The sum-product network case

TL;DR

This paper introduces ExSPN, an algorithm that converts Sum-Product Networks into interpretable CSI-trees, enhancing explainability of deep probabilistic models for domain experts.

Contribution

The paper presents a novel method to generate interpretable explanations for SPNs by converting them into CSI-trees, capturing context-specific independencies.

Findings

CSI-trees are more interpretable to domain experts.

The method effectively captures conditional independencies.

Empirical results show improved explainability on various datasets.

Abstract

We consider the problem of explaining a class of tractable deep probabilistic models, the Sum-Product Networks (SPNs) and present an algorithm ExSPN to generate explanations. To this effect, we define the notion of a context-specific independence tree(CSI-tree) and present an iterative algorithm that converts an SPN to a CSI-tree. The resulting CSI-tree is both interpretable and explainable to the domain expert. We achieve this by extracting the conditional independencies encoded by the SPN and approximating the local context specified by the structure of the SPN. Our extensive empirical evaluations on synthetic, standard, and real-world clinical data sets demonstrate that the CSI-tree exhibits superior explainability.