Improving the Explain-Any-Concept by Introducing Nonlinearity to the Trainable Surrogate Model

TL;DR

This paper enhances the Explain Any Concept (EAC) model in XAI by adding a nonlinear layer to the surrogate, leading to improved interpretability of deep neural networks in vision tasks on ImageNet and MS COCO datasets.

Contribution

Introducing a nonlinear layer into the EAC surrogate model to improve its performance in explaining DNN decisions.

Findings

Improved explanation accuracy on ImageNet dataset.

Enhanced interpretability on MS COCO dataset.

Nonlinear surrogate outperforms linear-only models.

Abstract

In the evolving field of Explainable AI (XAI), interpreting the decisions of deep neural networks (DNNs) in computer vision tasks is an important process. While pixel-based XAI methods focus on identifying significant pixels, existing concept-based XAI methods use pre-defined or human-annotated concepts. The recently proposed Segment Anything Model (SAM) achieved a significant step forward to prepare automatic concept sets via comprehensive instance segmentation. Building upon this, the Explain Any Concept (EAC) model emerged as a flexible method for explaining DNN decisions. EAC model is based on using a surrogate model which has one trainable linear layer to simulate the target model. In this paper, by introducing an additional nonlinear layer to the original surrogate model, we show that we can improve the performance of the EAC model. We compare our proposed approach to the original EAC model and report improvements obtained on both ImageNet and MS COCO datasets.