Explainable Image Recognition via Enhanced Slot-attention Based Classifier

TL;DR

This paper introduces ESCOUTER, a visually explainable image classifier that integrates explanations into confidence scores and provides positive or negative justifications, improving interpretability without sacrificing accuracy.

Contribution

ESCOUTER is a novel slot-attention based classifier that embeds explanations directly into decision confidence scores and offers comprehensive positive and negative explanations.

Findings

Outperforms previous state-of-the-art XAI methods across datasets

Provides intuitive explanations integrated with classification confidence

Effectively toggles between positive and negative explanations

Abstract

The imperative to comprehend the behaviors of deep learning models is of utmost importance. In this realm, Explainable Artificial Intelligence (XAI) has emerged as a promising avenue, garnering increasing interest in recent years. Despite this, most existing methods primarily depend on gradients or input perturbation, which often fails to embed explanations directly within the model's decision-making process. Addressing this gap, we introduce ESCOUTER, a visually explainable classifier based on the modified slot attention mechanism. ESCOUTER distinguishes itself by not only delivering high classification accuracy but also offering more transparent insights into the reasoning behind its decisions. It differs from prior approaches in two significant aspects: (a) ESCOUTER incorporates explanations into the final confidence scores for each category, providing a more intuitive interpretation, and (b) it offers positive or negative explanations for all categories, elucidating "why an image belongs to a certain category" or "why it does not." A novel loss function specifically for ESCOUTER is designed to fine-tune the model's behavior, enabling it to toggle between positive and negative explanations. Moreover, an area loss is also designed to adjust the size of the explanatory regions for a more precise explanation. Our method, rigorously tested across various datasets and XAI metrics, outperformed previous state-of-the-art methods, solidifying its effectiveness as an explanatory tool.