Neural Network Attributions: A Causal Perspective

TL;DR

This paper introduces a novel causal inference-based attribution method for neural networks, modeling the network as a Structural Causal Model to compute feature effects more accurately.

Contribution

It presents the first causal perspective-based attribution method for neural networks, including algorithms for efficient computation and scalability to high-dimensional data.

Findings

Effective on both simulated and real datasets

Applicable to recurrent neural networks

Shows promise in accurately attributing feature effects

Abstract

We propose a new attribution method for neural networks developed using first principles of causality (to the best of our knowledge, the first such). The neural network architecture is viewed as a Structural Causal Model, and a methodology to compute the causal effect of each feature on the output is presented. With reasonable assumptions on the causal structure of the input data, we propose algorithms to efficiently compute the causal effects, as well as scale the approach to data with large dimensionality. We also show how this method can be used for recurrent neural networks. We report experimental results on both simulated and real datasets showcasing the promise and usefulness of the proposed algorithm.