Representational Difference Explanations

TL;DR

The paper introduces Representational Differences Explanations (RDX), a novel method for visualizing and understanding differences between learned models' internal representations, improving interpretability and comparison in machine learning.

Contribution

RDX is a new technique that effectively visualizes and compares differences in learned representations, outperforming existing explainable AI methods in revealing meaningful distinctions.

Findings

RDX successfully recovers known conceptual differences between models.

It reveals subtle patterns and meaningful distinctions in complex datasets.

RDX outperforms existing XAI techniques in model comparison tasks.

Abstract

We propose a method for discovering and visualizing the differences between two learned representations, enabling more direct and interpretable model comparisons. We validate our method, which we call Representational Differences Explanations (RDX), by using it to compare models with known conceptual differences and demonstrate that it recovers meaningful distinctions where existing explainable AI (XAI) techniques fail. Applied to state-of-the-art models on challenging subsets of the ImageNet and iNaturalist datasets, RDX reveals both insightful representational differences and subtle patterns in the data. Although comparison is a cornerstone of scientific analysis, current tools in machine learning, namely post hoc XAI methods, struggle to support model comparison effectively. Our work addresses this gap by introducing an effective and explainable tool for contrasting model representations.