Fidelity of Ensemble Aggregation for Saliency Map Explanations using Bayesian Optimization Techniques

TL;DR

This paper explores ensemble aggregation of saliency maps for neural network explanations in computer vision, using Bayesian Optimization to improve fidelity by combining multiple explanations and analyzing normalization effects.

Contribution

It introduces a Bayesian Optimization-based method for aggregating saliency maps to enhance explanation fidelity and compares different normalization techniques.

Findings

Aggregation improves explanation fidelity over individual methods.

Bayesian Optimization effectively incorporates variance between explanations.

Normalization techniques influence aggregation quality.

Abstract

In recent years, an abundance of feature attribution methods for explaining neural networks have been developed. Especially in the field of computer vision, many methods for generating saliency maps providing pixel attributions exist. However, their explanations often contradict each other and it is not clear which explanation to trust. A natural solution to this problem is the aggregation of multiple explanations. We present and compare different pixel-based aggregation schemes with the goal of generating a new explanation, whose fidelity to the model's decision is higher than each individual explanation. Using methods from the field of Bayesian Optimization, we incorporate the variance between the individual explanations into the aggregation process. Additionally, we analyze the effect of multiple normalization techniques on ensemble aggregation.