BayesNAM: Leveraging Inconsistency for Reliable Explanations

TL;DR

This paper introduces BayesNAM, a Bayesian neural additive model that leverages explanation inconsistencies to identify data issues and improve interpretability in neural additive models.

Contribution

It presents a novel framework that uses Bayesian neural networks and feature dropout to harness explanation inconsistencies for more reliable AI explanations.

Findings

BayesNAM effectively reveals data problems like insufficient data.

Theoretical proof shows feature dropout captures model inconsistencies.

Experiments demonstrate improved explanation reliability.

Abstract

Neural additive model (NAM) is a recently proposed explainable artificial intelligence (XAI) method that utilizes neural network-based architectures. Given the advantages of neural networks, NAMs provide intuitive explanations for their predictions with high model performance. In this paper, we analyze a critical yet overlooked phenomenon: NAMs often produce inconsistent explanations, even when using the same architecture and dataset. Traditionally, such inconsistencies have been viewed as issues to be resolved. However, we argue instead that these inconsistencies can provide valuable explanations within the given data model. Through a simple theoretical framework, we demonstrate that these inconsistencies are not mere artifacts but emerge naturally in datasets with multiple important features. To effectively leverage this information, we introduce a novel framework, Bayesian Neural Additive Model (BayesNAM), which integrates Bayesian neural networks and feature dropout, with theoretical proof demonstrating that feature dropout effectively captures model inconsistencies. Our experiments demonstrate that BayesNAM effectively reveals potential problems such as insufficient data or structural limitations of the model, providing more reliable explanations and potential remedies.