Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations

TL;DR

This study compares human and GPT-3 explanations for data labels across easy and hard samples in the Winograd Schema Challenge, revealing that humans outperform models on hard examples in supportiveness and generalizability.

Contribution

It provides a detailed analysis of how explanation quality varies with sample hardness for both humans and GPT-3, highlighting the limitations of current LLM explanations on difficult samples.

Findings

GPT-3 explanations are as grammatical as human explanations regardless of sample hardness.

Humans provide more generalizable explanations for easy samples, while GPT-3 excels in supportiveness.

Humans outperform GPT-3 in explanation quality on hard samples.

Abstract

Recent work on explainable NLP has shown that few-shot prompting can enable large pretrained language models (LLMs) to generate grammatical and factual natural language explanations for data labels. In this work, we study the connection between explainability and sample hardness by investigating the following research question - "Are LLMs and humans equally good at explaining data labels for both easy and hard samples?" We answer this question by first collecting human-written explanations in the form of generalizable commonsense rules on the task of Winograd Schema Challenge (Winogrande dataset). We compare these explanations with those generated by GPT-3 while varying the hardness of the test samples as well as the in-context samples. We observe that (1) GPT-3 explanations are as grammatical as human explanations regardless of the hardness of the test samples, (2) for easy examples, GPT-3 generates highly supportive explanations but human explanations are more generalizable, and (3) for hard examples, human explanations are significantly better than GPT-3 explanations both in terms of label-supportiveness and generalizability judgements. We also find that hardness of the in-context examples impacts the quality of GPT-3 explanations. Finally, we show that the supportiveness and generalizability aspects of human explanations are also impacted by sample hardness, although by a much smaller margin than models. Supporting code and data are available at https://github.com/swarnaHub/ExplanationHardness