An Analysis of LIME for Text Data

TL;DR

This paper provides the first theoretical analysis of LIME for text data, confirming its effectiveness for simple models like decision trees and linear models, and highlighting the need for further guarantees.

Contribution

It offers a novel theoretical understanding of LIME's behavior on text data, which was previously lacking.

Findings

LIME provides meaningful explanations for decision trees.

LIME accurately explains linear models.

Theoretical guarantees for LIME are established for simple models.

Abstract

Text data are increasingly handled in an automated fashion by machine learning algorithms. But the models handling these data are not always well-understood due to their complexity and are more and more often referred to as "black-boxes." Interpretability methods aim to explain how these models operate. Among them, LIME has become one of the most popular in recent years. However, it comes without theoretical guarantees: even for simple models, we are not sure that LIME behaves accurately. In this paper, we provide a first theoretical analysis of LIME for text data. As a consequence of our theoretical findings, we show that LIME indeed provides meaningful explanations for simple models, namely decision trees and linear models.