Seeking Interpretability and Explainability in Binary Activated Neural Networks

TL;DR

This paper explores binary activated neural networks for regression on tabular data, emphasizing interpretability through guarantees on expressiveness and efficient SHAP value computation, and introduces a greedy algorithm for constructing compact, task-specific models.

Contribution

It introduces a method for building interpretable binary neural networks with guaranteed expressiveness and an efficient approach for computing feature importance via SHAP values.

Findings

Binary networks can be expressive enough for regression tasks.

The proposed greedy algorithm constructs compact, interpretable models.

Efficient SHAP value computation aids interpretability.

Abstract

We study the use of binary activated neural networks as interpretable and explainable predictors in the context of regression tasks on tabular data; more specifically, we provide guarantees on their expressiveness, present an approach based on the efficient computation of SHAP values for quantifying the relative importance of the features, hidden neurons and even weights. As the model's simplicity is instrumental in achieving interpretability, we propose a greedy algorithm for building compact binary activated networks. This approach doesn't need to fix an architecture for the network in advance: it is built one layer at a time, one neuron at a time, leading to predictors that aren't needlessly complex for a given task.