On the Granularity of Explanations in Model Agnostic NLP Interpretability

TL;DR

This paper proposes using sentence segments instead of individual words for interpreting BERT-based NLP models, improving fidelity and addressing limitations of current word-based explanation methods like LIME and SHAP.

Contribution

It introduces a segmentation-based approach for model-agnostic NLP interpretability, demonstrating significant improvements over word-based methods.

Findings

Sentence segmentation improves explanation fidelity.

Segment-based explanations are more computationally efficient.

The approach outperforms traditional word-based methods on benchmark tasks.

Abstract

Current methods for Black-Box NLP interpretability, like LIME or SHAP, are based on altering the text to interpret by removing words and modeling the Black-Box response. In this paper, we outline limitations of this approach when using complex BERT-based classifiers: The word-based sampling produces texts that are out-of-distribution for the classifier and further gives rise to a high-dimensional search space, which can't be sufficiently explored when time or computation power is limited. Both of these challenges can be addressed by using segments as elementary building blocks for NLP interpretability. As illustration, we show that the simple choice of sentences greatly improves on both of these challenges. As a consequence, the resulting explainer attains much better fidelity on a benchmark classification task.