Sanity Simulations for Saliency Methods

TL;DR

This paper introduces SMERF, a synthetic benchmarking framework for evaluating saliency methods against ground-truth model reasoning, revealing significant limitations in current techniques.

Contribution

The paper presents SMERF, a novel synthetic benchmark enabling ground-truth evaluation of saliency methods with controlled complexity.

Findings

Existing saliency methods have significant limitations.

SMERF effectively evaluates and compares saliency methods.

The framework aids in developing improved explanation techniques.

Abstract

Saliency methods are a popular class of feature attribution explanation methods that aim to capture a model's predictive reasoning by identifying "important" pixels in an input image. However, the development and adoption of these methods are hindered by the lack of access to ground-truth model reasoning, which prevents accurate evaluation. In this work, we design a synthetic benchmarking framework, SMERF, that allows us to perform ground-truth-based evaluation while controlling the complexity of the model's reasoning. Experimentally, SMERF reveals significant limitations in existing saliency methods and, as a result, represents a useful tool for the development of new saliency methods.