# The Generalizability of Explanations

## TL;DR

This paper introduces a new evaluation method for explainability techniques based on their generalizability, using an Autoencoder to assess the learnability and plausibility of explanations, and shows smoothing improves this property.

## Contribution

It proposes a novel generalizability-based evaluation framework for explainability methods utilizing Autoencoders, addressing the lack of ground truth in explainability evaluation.

## Key findings

- Smoothing explanations with SmoothGrad improves their generalizability.
- The Autoencoder-based approach effectively evaluates multiple explainability methods.
- The methodology provides a new perspective on explainability evaluation beyond human and sensitivity tests.

## Abstract

Due to the absence of ground truth, objective evaluation of explainability methods is an essential research direction. So far, the vast majority of evaluations can be summarized into three categories, namely human evaluation, sensitivity testing, and salinity check. This work proposes a novel evaluation methodology from the perspective of generalizability. We employ an Autoencoder to learn the distributions of the generated explanations and observe their learnability as well as the plausibility of the learned distributional features. We first briefly demonstrate the evaluation idea of the proposed approach at LIME, and then quantitatively evaluate multiple popular explainability methods. We also find that smoothing the explanations with SmoothGrad can significantly enhance the generalizability of explanations.