IMACS: Image Model Attribution Comparison Summaries

TL;DR

IMACS is a framework that compares and visualizes differences in model attributions between two image models, aiding interpretability and understanding of model behavior across datasets.

Contribution

We introduce IMACS, a novel method that aggregates and visualizes attribution differences between models, enhancing interpretability in image classification tasks.

Findings

IMACS effectively highlights behavioral differences between models.

The method uncovers domain shift effects in satellite image models.

Visual summaries facilitate understanding of attribution variations.

Abstract

Developing a suitable Deep Neural Network (DNN) often requires significant iteration, where different model versions are evaluated and compared. While metrics such as accuracy are a powerful means to succinctly describe a model's performance across a dataset or to directly compare model versions, practitioners often wish to gain a deeper understanding of the factors that influence a model's predictions. Interpretability techniques such as gradient-based methods and local approximations can be used to examine small sets of inputs in fine detail, but it can be hard to determine if results from small sets generalize across a dataset. We introduce IMACS, a method that combines gradient-based model attributions with aggregation and visualization techniques to summarize differences in attributions between two DNN image models. More specifically, IMACS extracts salient input features from an evaluation dataset, clusters them based on similarity, then visualizes differences in model attributions for similar input features. In this work, we introduce a framework for aggregating, summarizing, and comparing the attribution information for two models across a dataset; present visualizations that highlight differences between 2 image classification models; and show how our technique can uncover behavioral differences caused by domain shift between two models trained on satellite images.