Factuality Enhanced Language Models for Open-Ended Text Generation

TL;DR

This paper introduces methods to measure and enhance the factual accuracy of large language models in open-ended text generation, including new benchmarks, sampling algorithms, and training techniques that significantly reduce factual errors.

Contribution

It presents the FactualityPrompts benchmark, a factual-nucleus sampling algorithm, and a factuality-enhanced training method to improve the factual correctness of language models.

Findings

Larger LMs tend to be more factual than smaller ones.

Factual-nucleus sampling improves factuality without sacrificing quality.

Factuality-enhanced training reduces factual errors significantly.

Abstract

Pretrained language models (LMs) are susceptible to generate text with nonfactual information. In this work, we measure and improve the factual accuracy of large-scale LMs for open-ended text generation. We design the FactualityPrompts test set and metrics to measure the factuality of LM generations. Based on that, we study the factual accuracy of LMs with parameter sizes ranging from 126M to 530B. Interestingly, we find that larger LMs are more factual than smaller ones, although a previous study suggests that larger LMs can be less truthful in terms of misconceptions. In addition, popular sampling algorithms (e.g., top-p) in open-ended text generation can harm the factuality due to the ''uniform randomness'' introduced at every sampling step. We propose the factual-nucleus sampling algorithm that dynamically adapts the randomness to improve the factuality of generation while maintaining quality. Furthermore, we analyze the inefficiencies of the standard training method in learning correct associations between entities from factual text corpus (e.g., Wikipedia). We propose a factuality-enhanced training method that uses TopicPrefix for better awareness of facts and sentence completion as the training objective, which can vastly reduce the factual errors. We release our code and FactualityPrompts benchmark at: https://github.com/nayeon7lee/FactualityPrompt.