For Generated Text, Is NLI-Neutral Text the Best Text?
Michail Mersinias, Kyle Mahowald

TL;DR
This paper investigates using a pre-trained NLI model to improve text generation quality by assessing entailment, contradiction, or neutrality, and finds that maximizing neutrality yields the best results across different sampling settings.
Contribution
It introduces an NLI-informed approach to enhance GPT-based text generation by leveraging entailment, contradiction, and neutrality assessments, demonstrating improved quality over standard methods.
Findings
Maximizing entailment improves generation at high randomness settings.
Maximizing contradiction is effective at low randomness.
Maximizing neutrality consistently yields the highest quality text.
Abstract
We explore incorporating natural language inference (NLI) into the text generative pipeline by using a pre-trained NLI model to assess whether a generated sentence entails, contradicts, or is neutral to the prompt and preceding text. First, we show that the NLI task is predictive of generation errors made by GPT-3. We use these results to develop an NLI-informed generation procedure for GPT-J. Then, we evaluate these generations by obtaining human annotations on error types and overall quality. We find that an NLI strategy of maximizing entailment improves text generation when the nucleus sampling randomness parameter value is high, while one which maximizes contradiction is in fact productive when the parameter value is low. Overall, though, we demonstrate that an NLI strategy of maximizing the neutral class provides the highest quality of generated text (significantly better than the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
MethodsMulti-Head Attention · Attention Is All You Need · Cosine Annealing · Linear Layer · Dense Connections · Weight Decay · Adam · Linear Warmup With Cosine Annealing · 15 Ways to Contact How can i speak to someone at Delta Airlines · {Dispute@FaQ-s}How to file a dispute with Expedia?
