Sampling Based On Natural Image Statistics Improves Local Surrogate Explainers

TL;DR

This paper proposes new sampling methods for local surrogate explainers in computer vision that incorporate natural image statistics, leading to more accurate and meaningful explanations of deep neural network predictions.

Contribution

It introduces two approaches to align local sampling with natural image data distribution, improving surrogate explanations without needing access to the original training data.

Findings

Sampling based on natural image statistics enhances explanation quality.

Perceptual metrics improve the relevance of local neighborhoods.

Methods outperform traditional sampling in local surrogate explanations.

Abstract

Many problems in computer vision have recently been tackled using models whose predictions cannot be easily interpreted, most commonly deep neural networks. Surrogate explainers are a popular post-hoc interpretability method to further understand how a model arrives at a particular prediction. By training a simple, more interpretable model to locally approximate the decision boundary of a non-interpretable system, we can estimate the relative importance of the input features on the prediction. Focusing on images, surrogate explainers, e.g., LIME, generate a local neighbourhood around a query image by sampling in an interpretable domain. However, these interpretable domains have traditionally been derived exclusively from the intrinsic features of the query image, not taking into consideration the manifold of the data the non-interpretable model has been exposed to in training (or more generally, the manifold of real images). This leads to suboptimal surrogates trained on potentially low probability images. We address this limitation by aligning the local neighbourhood on which the surrogate is trained with the original training data distribution, even when this distribution is not accessible. We propose two approaches to do so, namely (1) altering the method for sampling the local neighbourhood and (2) using perceptual metrics to convey some of the properties of the distribution of natural images.