HINT: Hierarchical Neuron Concept Explainer

TL;DR

HINT is a scalable method for interpreting deep neural networks by associating neurons with hierarchical concepts, revealing how neurons encode both concrete and abstract concepts and their relationships.

Contribution

The paper introduces HINT, a novel approach that models hierarchical concept associations in neurons, enabling systematic analysis of concept embedding in deep networks.

Findings

Identifies collaborative neurons responsible for specific concepts.

Reveals multimodal neurons for different concepts.

Demonstrates applicability in saliency detection and adversarial explanation.

Abstract

To interpret deep networks, one main approach is to associate neurons with human-understandable concepts. However, existing methods often ignore the inherent relationships of different concepts (e.g., dog and cat both belong to animals), and thus lose the chance to explain neurons responsible for higher-level concepts (e.g., animal). In this paper, we study hierarchical concepts inspired by the hierarchical cognition process of human beings. To this end, we propose HIerarchical Neuron concepT explainer (HINT) to effectively build bidirectional associations between neurons and hierarchical concepts in a low-cost and scalable manner. HINT enables us to systematically and quantitatively study whether and how the implicit hierarchical relationships of concepts are embedded into neurons, such as identifying collaborative neurons responsible to one concept and multimodal neurons for different concepts, at different semantic levels from concrete concepts (e.g., dog) to more abstract ones (e.g., animal). Finally, we verify the faithfulness of the associations using Weakly Supervised Object Localization, and demonstrate its applicability in various tasks such as discovering saliency regions and explaining adversarial attacks. Code is available on https://github.com/AntonotnaWang/HINT.