A Super-pixel-based Approach to the Stable Interpretation of Neural Networks

TL;DR

This paper introduces a super-pixel-based method to enhance the stability and interpretability of saliency maps in neural network visualization, reducing stochasticity and aligning better with image semantics.

Contribution

It proposes a novel super-pixel grouping strategy that improves the stability and generalization of gradient-based saliency maps for neural network interpretation.

Findings

Super-pixel grouping reduces saliency map variance.

Method improves stability over pixel-based saliency maps.

Experimental results on CIFAR-10 and ImageNet confirm effectiveness.

Abstract

Saliency maps are widely used in the computer vision community for interpreting neural network classifiers. However, due to the randomness of training samples and optimization algorithms, the resulting saliency maps suffer from a significant level of stochasticity, making it difficult for domain experts to capture the intrinsic factors that influence the neural network's decision. In this work, we propose a novel pixel partitioning strategy to boost the stability and generalizability of gradient-based saliency maps. Through both theoretical analysis and numerical experiments, we demonstrate that the grouping of pixels reduces the variance of the saliency map and improves the generalization behavior of the interpretation method. Furthermore, we propose a sensible grouping strategy based on super-pixels which cluster pixels into groups that align well with the semantic meaning of the images. We perform several numerical experiments on CIFAR-10 and ImageNet. Our empirical results suggest that the super-pixel-based interpretation maps consistently improve the stability and quality over the pixel-based saliency maps.