Unsupervised Causal Binary Concepts Discovery with VAE for Black-box Model Explanation

TL;DR

This paper introduces an unsupervised method using a VAE-based structural generative model to discover high-level binary concepts that explain black-box classifiers, with the ability to incorporate user prior knowledge.

Contribution

It presents a novel unsupervised approach combining a structural generative model and causal influence learning to identify meaningful concepts for model explanation.

Findings

Successfully discovers binary concepts across multiple datasets.

Enhances interpretability by integrating user prior knowledge.

Effectively explains black-box classifiers with high-level concepts.

Abstract

We aim to explain a black-box classifier with the form: `data X is classified as class Y because X \textit{has} A, B and \textit{does not have} C' in which A, B, and C are high-level concepts. The challenge is that we have to discover in an unsupervised manner a set of concepts, i.e., A, B and C, that is useful for the explaining the classifier. We first introduce a structural generative model that is suitable to express and discover such concepts. We then propose a learning process that simultaneously learns the data distribution and encourages certain concepts to have a large causal influence on the classifier output. Our method also allows easy integration of user's prior knowledge to induce high interpretability of concepts. Using multiple datasets, we demonstrate that our method can discover useful binary concepts for explanation.