Faithful to the Document or to the World? Mitigating Hallucinations via Entity-linked Knowledge in Abstractive Summarization

TL;DR

This paper explores how integrating external entity-linked knowledge bases into abstractive summarization models can reduce hallucinations and improve factual accuracy by leveraging external world knowledge and reasoning paths.

Contribution

It introduces a method to incorporate external knowledge bases to enhance faithfulness and factuality in abstractive summarization, addressing limitations of source-only models.

Findings

External knowledge improves summary faithfulness.

Knowledge-based methods reduce hallucinations.

Enhanced factual accuracy with linked knowledge bases.

Abstract

Despite recent advances in abstractive summarization, current summarization systems still suffer from content hallucinations where models generate text that is either irrelevant or contradictory to the source document. However, prior work has been predicated on the assumption that any generated facts not appearing explicitly in the source are undesired hallucinations. Methods have been proposed to address this scenario by ultimately improving `faithfulness' to the source document, but in reality, there is a large portion of entities in the gold reference targets that are not directly in the source. In this work, we show that these entities are not aberrations, but they instead require utilizing external world knowledge to infer reasoning paths from entities in the source. We show that by utilizing an external knowledge base, we can improve the faithfulness of summaries without simply making them more extractive, and additionally, we show that external knowledge bases linked from the source can benefit the factuality of generated summaries.