Locally Testing Model Detections for Semantic Global Concepts

TL;DR

This paper introduces a framework called glCA that tests DNNs for specific semantic concepts at a local level, linking global concept encodings to individual input processing to improve model interpretability and safety.

Contribution

The paper presents a novel global-to-local concept attribution method that conditions local explanations on predefined semantic concepts in the model's latent space.

Findings

Major differences in local perception of global concepts were identified.

The approach enables detailed understanding of concept encoding within models.

Further research is needed to improve semantic concept encodings.

Abstract

Ensuring the quality of black-box Deep Neural Networks (DNNs) has become ever more significant, especially in safety-critical domains such as automated driving. While global concept encodings generally enable a user to test a model for a specific concept, linking global concept encodings to the local processing of single network inputs reveals their strengths and limitations. Our proposed framework global-to-local Concept Attribution (glCA) uses approaches from local (why a specific prediction originates) and global (how a model works generally) eXplainable Artificial Intelligence (xAI) to test DNNs for a predefined semantical concept locally. The approach allows for conditioning local, post-hoc explanations on predefined semantic concepts encoded as linear directions in the model's latent space. Pixel-exact scoring concerning the global concept usage assists the tester in further understanding the model processing of single data points for the selected concept. Our approach has the advantage of fully covering the model-internal encoding of the semantic concept and allowing the localization of relevant concept-related information. The results show major differences in the local perception and usage of individual global concept encodings and demand for further investigations regarding obtaining thorough semantic concept encodings.