Forward Composition Propagation for Explainable Neural Reasoning

TL;DR

This paper introduces Forward Composition Propagation (FCP), an algorithm for explaining neural network predictions by propagating composition vectors that reveal feature influences, demonstrated through a bias detection case study.

Contribution

The paper presents a novel FCP algorithm that explains neural network decisions by propagating composition vectors, providing interpretability for structured classification tasks.

Findings

Composition vectors align with expected feature roles in bias detection

FCP provides clear insights into feature excitation and inhibition

Source code is publicly available for further use

Abstract

This paper proposes an algorithm called Forward Composition Propagation (FCP) to explain the predictions of feed-forward neural networks operating on structured classification problems. In the proposed FCP algorithm, each neuron is described by a composition vector indicating the role of each problem feature in that neuron. Composition vectors are initialized using a given input instance and subsequently propagated through the whole network until reaching the output layer. The sign of each composition value indicates whether the corresponding feature excites or inhibits the neuron, while the absolute value quantifies its impact. The FCP algorithm is executed on a post-hoc basis, i.e., once the learning process is completed. Aiming to illustrate the FCP algorithm, this paper develops a case study concerning bias detection in a fairness problem in which the ground truth is known. The simulation results show that the composition values closely align with the expected behavior of protected features. The source code and supplementary material for this paper are available at https://github.com/igraugar/fcp.