Towards credible visual model interpretation with path attribution

TL;DR

This paper improves the reliability of path attribution methods for visual model interpretation by identifying conditions to avoid counter-intuitive results and proposing a scheme to ensure axiomatic properties, validated through extensive experiments.

Contribution

It systematically analyzes issues in path attribution for deep visual models and introduces a method to enhance interpretability consistency and reliability.

Findings

Our method outperforms baselines across multiple datasets.

Counter-intuitive results can be mitigated with specific conditions.

The proposed scheme maintains axiomatic properties in visual interpretations.

Abstract

Originally inspired by game-theory, path attribution framework stands out among the post-hoc model interpretation tools due to its axiomatic nature. However, recent developments show that this framework can still suffer from counter-intuitive results. Moreover, specifically for deep visual models, the existing path-based methods also fall short on conforming to the original intuitions that are the basis of the claimed axiomatic properties of this framework. We address these problems with a systematic investigation, and pinpoint the conditions in which the counter-intuitive results can be avoided for deep visual model interpretation with the path attribution strategy. We also devise a scheme to preclude the conditions in which visual model interpretation can invalidate the axiomatic properties of path attribution. These insights are combined into a method that enables reliable visual model interpretation. Our findings are establish empirically with multiple datasets, models and evaluation metrics. Extensive experiments show a consistent performance gain of our method over the baselines.