LLMs for Generating and Evaluating Counterfactuals: A Comprehensive Study

TL;DR

This study evaluates how well Large Language Models generate and assess counterfactual explanations in NLP, revealing their strengths in fluency but limitations in minimality and label-flipping, with implications for data augmentation and model interpretability.

Contribution

It provides a comprehensive comparison of LLMs in generating and evaluating counterfactuals for NLP tasks, highlighting their capabilities and limitations.

Findings

LLMs generate fluent counterfactuals but struggle with minimal changes.

Generating counterfactuals for Sentiment Analysis is easier than for NLI.

LLMs show bias towards original labels, affecting evaluation and augmentation.

Abstract

As NLP models become more complex, understanding their decisions becomes more crucial. Counterfactuals (CFs), where minimal changes to inputs flip a model's prediction, offer a way to explain these models. While Large Language Models (LLMs) have shown remarkable performance in NLP tasks, their efficacy in generating high-quality CFs remains uncertain. This work fills this gap by investigating how well LLMs generate CFs for two NLU tasks. We conduct a comprehensive comparison of several common LLMs, and evaluate their CFs, assessing both intrinsic metrics, and the impact of these CFs on data augmentation. Moreover, we analyze differences between human and LLM-generated CFs, providing insights for future research directions. Our results show that LLMs generate fluent CFs, but struggle to keep the induced changes minimal. Generating CFs for Sentiment Analysis (SA) is less challenging than NLI where LLMs show weaknesses in generating CFs that flip the original label. This also reflects on the data augmentation performance, where we observe a large gap between augmenting with human and LLMs CFs. Furthermore, we evaluate LLMs' ability to assess CFs in a mislabelled data setting, and show that they have a strong bias towards agreeing with the provided labels. GPT4 is more robust against this bias and its scores correlate well with automatic metrics. Our findings reveal several limitations and point to potential future work directions.