Neural Networks Decoded: Targeted and Robust Analysis of Neural Network Decisions via Causal Explanations and Reasoning

TL;DR

TRACER is a causal inference-based method that interprets neural network decisions by analyzing input interventions, internal activations, and generating counterfactuals, improving transparency without affecting model performance.

Contribution

It introduces TRACER, a novel causal inference approach that provides structured, interpretable explanations of DNN decisions without altering the model architecture or accuracy.

Findings

TRACER outperforms existing interpretability methods in diverse datasets.

It effectively identifies feature importance and model biases.

TRACER enables creation of compressed yet accurate neural network models.

Abstract

Despite their success and widespread adoption, the opaque nature of deep neural networks (DNNs) continues to hinder trust, especially in critical applications. Current interpretability solutions often yield inconsistent or oversimplified explanations, or require model changes that compromise performance. In this work, we introduce TRACER, a novel method grounded in causal inference theory designed to estimate the causal dynamics underpinning DNN decisions without altering their architecture or compromising their performance. Our approach systematically intervenes on input features to observe how specific changes propagate through the network, affecting internal activations and final outputs. Based on this analysis, we determine the importance of individual features, and construct a high-level causal map by grouping functionally similar layers into cohesive causal nodes, providing a structured and interpretable view of how different parts of the network influence the decisions. TRACER further enhances explainability by generating counterfactuals that reveal possible model biases and offer contrastive explanations for misclassifications. Through comprehensive evaluations across diverse datasets, we demonstrate TRACER's effectiveness over existing methods and show its potential for creating highly compressed yet accurate models, illustrating its dual versatility in both understanding and optimizing DNNs.