Enhancing Model Interpretability with Local Attribution over Global Exploration

TL;DR

This paper introduces a Local Attribution (LA) algorithm that improves model interpretability by focusing on local space exploration, reducing the impact of out-of-distribution states, and outperforming existing attribution methods.

Contribution

The paper proposes a novel LA algorithm that leverages local space properties with targeted and untargeted exploration phases for better attribution accuracy.

Findings

Achieves 38.21% improvement in attribution effectiveness over state-of-the-art methods.

Effectively generates intermediate states within local space, avoiding OOD issues.

Extensive ablation studies confirm the importance of each component in the LA algorithm.

Abstract

In the field of artificial intelligence, AI models are frequently described as `black boxes' due to the obscurity of their internal mechanisms. It has ignited research interest on model interpretability, especially in attribution methods that offers precise explanations of model decisions. Current attribution algorithms typically evaluate the importance of each parameter by exploring the sample space. A large number of intermediate states are introduced during the exploration process, which may reach the model's Out-of-Distribution (OOD) space. Such intermediate states will impact the attribution results, making it challenging to grasp the relative importance of features. In this paper, we firstly define the local space and its relevant properties, and we propose the Local Attribution (LA) algorithm that leverages these properties. The LA algorithm comprises both targeted and untargeted exploration phases, which are designed to effectively generate intermediate states for attribution that thoroughly encompass the local space. Compared to the state-of-the-art attribution methods, our approach achieves an average improvement of 38.21\% in attribution effectiveness. Extensive ablation studies in our experiments also validate the significance of each component in our algorithm. Our code is available at: https://github.com/LMBTough/LA/