Inference Graphs for CNN Interpretation

TL;DR

This paper introduces inference graphs based on probabilistic models to interpret CNNs by visualizing layer activities and transitions, enhancing understanding of network decision processes.

Contribution

It presents a novel approach using Gaussian mixture models and transition probabilities to create inference graphs for CNN interpretation.

Findings

Inference graphs help visualize CNN decision pathways.

The method clarifies class-specific inference processes.

It improves trust and interpretability of CNNs.

Abstract

Convolutional neural networks (CNNs) have achieved superior accuracy in many visual related tasks. However, the inference process through intermediate layers is opaque, making it difficult to interpret such networks or develop trust in their operation. We propose to model the network hidden layers activity using probabilistic models. The activity patterns in layers of interest are modeled as Gaussian mixture models, and transition probabilities between clusters in consecutive modeled layers are estimated. Based on maximum-likelihood considerations, nodes and paths relevant for network prediction are chosen, connected, and visualized as an inference graph. We show that such graphs are useful for understanding the general inference process of a class, as well as explaining decisions the network makes regarding specific images.