Select Wisely and Explain: Active Learning and Probabilistic Local Post-hoc Explainability

TL;DR

This paper introduces UnRAvEL, an active learning method that improves local explanations of black-box models by using uncertainty-driven sampling with Gaussian process regression, enhancing stability and fidelity.

Contribution

The paper proposes UnRAvEL, a novel active learning approach for generating reliable local explanations, with theoretical analysis and demonstrated effectiveness on real-world datasets.

Findings

UnRAvEL outperforms baselines in stability and local fidelity.

UnRAvEL effectively generates surrogate datasets for explanation.

Demonstrated sample efficiency on ImageNet with ResNet.

Abstract

Albeit the tremendous performance improvements in designing complex artificial intelligence (AI) systems in data-intensive domains, the black-box nature of these systems leads to the lack of trustworthiness. Post-hoc interpretability methods explain the prediction of a black-box ML model for a single instance, and such explanations are being leveraged by domain experts to diagnose the underlying biases of these models. Despite their efficacy in providing valuable insights, existing approaches fail to deliver consistent and reliable explanations. In this paper, we propose an active learning-based technique called UnRAvEL (Uncertainty driven Robust Active Learning Based Locally Faithful Explanations), which consists of a novel acquisition function that is locally faithful and uses uncertainty-driven sampling based on the posterior distribution on the probabilistic locality using Gaussian process regression(GPR). We present a theoretical analysis of UnRAvEL by treating it as a local optimizer and analyzing its regret in terms of instantaneous regrets over a global optimizer. We demonstrate the efficacy of the local samples generated by UnRAvEL by incorporating different kernels such as the Matern and linear kernels in GPR. Through a series of experiments, we show that UnRAvEL outperforms the baselines with respect to stability and local fidelity on several real-world models and datasets. We show that UnRAvEL is an efficient surrogate dataset generator by deriving importance scores on this surrogate dataset using sparse linear models. We also showcase the sample efficiency and flexibility of the developed framework on the Imagenet dataset using a pre-trained ResNet model.