Explanation Strategies for Image Classification in Humans vs. Current Explainable AI

TL;DR

This study compares human explanation strategies in image classification with current XAI methods, revealing differences in information use and highlighting the importance of causality-based explanations for better human alignment.

Contribution

It identifies distinct human explanation strategies and shows that XAI saliency maps align more with explorative strategies and causality-based explanations, informing better XAI design.

Findings

Humans use explorative and focused attention strategies.

XAI saliency maps resemble human explorative strategies.

Causality-based explanations align better with human explanations.

Abstract

Explainable AI (XAI) methods provide explanations of AI models, but our understanding of how they compare with human explanations remains limited. In image classification, we found that humans adopted more explorative attention strategies for explanation than the classification task itself. Two representative explanation strategies were identified through clustering: One involved focused visual scanning on foreground objects with more conceptual explanations diagnostic for inferring class labels, whereas the other involved explorative scanning with more visual explanations rated higher for effectiveness. Interestingly, XAI saliency-map explanations had the highest similarity to the explorative attention strategy in humans, and explanations highlighting discriminative features from invoking observable causality through perturbation had higher similarity to human strategies than those highlighting internal features associated with higher class score. Thus, humans differ in information and strategy use for explanations, and XAI methods that highlight features informing observable causality match better with human explanations, potentially more accessible to users.