An Interpretable Probabilistic Approach for Demystifying Black-box Predictive Models

TL;DR

This paper introduces a Bayesian network-based method for post hoc interpretation of black-box models, providing insights into feature contributions, focused explanations, and confidence assessment, enhancing interpretability of complex machine learning models.

Contribution

The paper presents a novel Bayesian network framework that offers detailed feature contribution explanations, focused interpretations via Markov blankets, and confidence rules for black-box models.

Findings

Effective extraction of Bayesian networks approximating black-box models

Provides explanations of feature importance and contribution

Enables confidence assessment through derived rules

Abstract

The use of sophisticated machine learning models for critical decision making is faced with a challenge that these models are often applied as a "black-box". This has led to an increased interest in interpretable machine learning, where post hoc interpretation presents a useful mechanism for generating interpretations of complex learning models. In this paper, we propose a novel approach underpinned by an extended framework of Bayesian networks for generating post hoc interpretations of a black-box predictive model. The framework supports extracting a Bayesian network as an approximation of the black-box model for a specific prediction. Compared to the existing post hoc interpretation methods, the contribution of our approach is three-fold. Firstly, the extracted Bayesian network, as a probabilistic graphical model, can provide interpretations about not only what input features but also why these features contributed to a prediction. Secondly, for complex decision problems with many features, a Markov blanket can be generated from the extracted Bayesian network to provide interpretations with a focused view on those input features that directly contributed to a prediction. Thirdly, the extracted Bayesian network enables the identification of four different rules which can inform the decision-maker about the confidence level in a prediction, thus helping the decision-maker assess the reliability of predictions learned by a black-box model. We implemented the proposed approach, applied it in the context of two well-known public datasets and analysed the results, which are made available in an open-source repository.