Explainable AI Approach using Near Misses Analysis

TL;DR

This paper presents a novel explainable AI method based on near-misses analysis that uncovers hierarchical concepts in neural networks, applicable across various architectures and datasets, and introduces a new explainability metric.

Contribution

The paper introduces a new NMA-based XAI approach that reveals hierarchical concepts without analyzing explicit network structure, applicable to multiple architectures and datasets.

Findings

NMA reflects neural network latent concept generation

Efficient architectures may sacrifice explainability and robustness

New metric for measuring explainability

Abstract

This paper introduces a novel XAI approach based on near-misses analysis (NMA). This approach reveals a hierarchy of logical 'concepts' inferred from the latent decision-making process of a Neural Network (NN) without delving into its explicit structure. We examined our proposed XAI approach on different network architectures that vary in size and shape (e.g., ResNet, VGG, EfficientNet, MobileNet) on several datasets (ImageNet and CIFAR100). The results demonstrate its usability to reflect NNs latent process of concepts generation. We generated a new metric for explainability. Moreover, our experiments suggest that efficient architectures, which achieve a similar accuracy level with much less neurons may still pay the price of explainability and robustness in terms of concepts generation. We, thus, pave a promising new path for XAI research to follow.