A General Search-based Framework for Generating Textual Counterfactual Explanations

TL;DR

This paper introduces a flexible, search-based method for generating textual counterfactual explanations that is adaptable to various models and domains without retraining, addressing limitations of generative models.

Contribution

It proposes a model-agnostic, domain-agnostic search framework for counterfactual text generation that requires no retraining to accommodate changing user requirements.

Findings

Framework is model-agnostic and domain-agnostic.

Can incorporate domain-specific knowledge through specialized operators.

Does not require retraining when requirements change.

Abstract

One of the prominent methods for explaining the decision of a machine-learning classifier is by a counterfactual example. Most current algorithms for generating such examples in the textual domain are based on generative language models. Generative models, however, are trained to minimize a specific loss function in order to fulfill certain requirements for the generated texts. Any change in the requirements may necessitate costly retraining, thus potentially limiting their applicability. In this paper, we present a general search-based framework for generating counterfactual explanations in the textual domain. Our framework is model-agnostic, domain-agnostic, anytime, and does not require retraining in order to adapt to changes in the user requirements. We model the task as a search problem in a space where the initial state is the classified text, and the goal state is a text in a given target class. Our framework includes domain-independent modification operators, but can also exploit domain-specific knowledge through specialized operators. The search algorithm attempts to find a text from the target class with minimal user-specified distance from the original classified object.