Semantic Network Interpretation

TL;DR

This paper introduces a semantic network interpretation framework that combines filter-level visual attribute distributions and decision-level textual summaries to better understand neural network decisions and failure patterns.

Contribution

It presents a novel Bayesian inference method for automatic association of filters and decisions with visual attributes, enhancing interpretability beyond visualization.

Findings

Semantic interpretation aids in understanding network failures.

Correlation between model performance and distribution metrics is clarified.

Human study confirms the usefulness of semantic interpretation.

Abstract

Network interpretation as an effort to reveal the features learned by a network remains largely visualization-based. In this paper, our goal is to tackle semantic network interpretation at both filter and decision level. For filter-level interpretation, we represent the concepts a filter encodes with a probability distribution of visual attributes. The decision-level interpretation is achieved by textual summarization that generates an explanatory sentence containing clues behind a network's decision. A Bayesian inference algorithm is proposed to automatically associate filters and network decisions with visual attributes. Human study confirms that the semantic interpretation is a beneficial alternative or complement to visualization methods. We demonstrate the crucial role that semantic network interpretation can play in understanding a network's failure patterns. More importantly, semantic network interpretation enables a better understanding of the correlation between a model's performance and its distribution metrics like filter selectivity and concept sparseness.