EMaP: Explainable AI with Manifold-based Perturbations

TL;DR

This paper introduces EMaP, a novel perturbation scheme for explainable AI that perturbs data along orthogonal directions to the data manifold, enhancing explanation faithfulness and robustness against attacks.

Contribution

The paper proposes a new orthogonal perturbation method, EMaP, improving explanation quality and robustness in perturbation-based explainable AI methods.

Findings

Orthogonal perturbations better preserve data topology.

EMaP improves explanation performance.

EMaP enhances robustness against perturbation attacks.

Abstract

In the last few years, many explanation methods based on the perturbations of input data have been introduced to improve our understanding of decisions made by black-box models. The goal of this work is to introduce a novel perturbation scheme so that more faithful and robust explanations can be obtained. Our study focuses on the impact of perturbing directions on the data topology. We show that perturbing along the orthogonal directions of the input manifold better preserves the data topology, both in the worst-case analysis of the discrete Gromov-Hausdorff distance and in the average-case analysis via persistent homology. From those results, we introduce EMaP algorithm, realizing the orthogonal perturbation scheme. Our experiments show that EMaP not only improves the explainers' performance but also helps them overcome a recently-developed attack against perturbation-based methods.