Hypothesis-only Biases in Large Language Model-Elicited Natural Language Inference

TL;DR

This paper investigates whether large language models produce NLI hypotheses with annotation artifacts similar to crowdsourced data, revealing persistent biases and frequent tell-tale phrases in LLM-generated hypotheses.

Contribution

It demonstrates that LLM-generated NLI hypotheses contain significant artifacts, challenging assumptions about bias reduction in automated data creation.

Findings

Hypotheses from LLMs achieve 86-96% accuracy with hypothesis-only classifiers.

LLM-generated hypotheses contain frequent, identifiable bias phrases.

Biases in NLI data persist even when hypotheses are generated by LLMs.

Abstract

We test whether replacing crowdsource workers with LLMs to write Natural Language Inference (NLI) hypotheses similarly results in annotation artifacts. We recreate a portion of the Stanford NLI corpus using GPT-4, Llama-2 and Mistral 7b, and train hypothesis-only classifiers to determine whether LLM-elicited hypotheses contain annotation artifacts. On our LLM-elicited NLI datasets, BERT-based hypothesis-only classifiers achieve between 86-96% accuracy, indicating these datasets contain hypothesis-only artifacts. We also find frequent "give-aways" in LLM-generated hypotheses, e.g. the phrase "swimming in a pool" appears in more than 10,000 contradictions generated by GPT-4. Our analysis provides empirical evidence that well-attested biases in NLI can persist in LLM-generated data.