InFact: Informativeness Alignment for Improved LLM Factuality

TL;DR

This paper introduces InFact, an informativeness alignment mechanism that enhances the factuality and informativeness of large language models by prioritizing correct and detailed responses.

Contribution

It proposes a novel informativeness alignment objective that improves both factual correctness and informativeness in LLM outputs.

Findings

Improved factuality and informativeness through the alignment mechanism

Enhanced model performance on factual benchmarks

Better balance between correctness and detail in generated text

Abstract

Factual completeness is a general term that captures how detailed and informative a factually correct text is. For instance, the factual sentence ``Barack Obama was born in the United States'' is factually correct, though less informative than the factual sentence ``Barack Obama was born in Honolulu, Hawaii, United States''. Despite the known fact that LLMs tend to hallucinate and generate factually incorrect text, they might also tend to choose to generate factual text that is indeed factually correct and yet less informative than other, more informative choices. In this work, we tackle this problem by proposing an informativeness alignment mechanism. This mechanism takes advantage of recent factual benchmarks to propose an informativeness alignment objective. This objective prioritizes answers that are both correct and informative. A key finding of our work is that when training a model to maximize this objective or optimize its preference, we can improve not just informativeness but also factuality.