Improving Factuality for Dialogue Response Generation via Graph-Based Knowledge Augmentation

TL;DR

This paper introduces two graph-based knowledge augmentation frameworks to enhance factual accuracy in dialogue response generation, addressing hallucination issues in large language models.

Contribution

It proposes novel graph knowledge-augmented frameworks and a dialogue fact score for more reliable factuality assessment, improving over existing methods.

Findings

Significant improvement in factuality scores on OpendialKG and HybriDialogue datasets.

Introduction of a dialogue-specific fact score for better factuality evaluation.

Outperforms state-of-the-art graph knowledge-augmentation baselines.

Abstract

Large Language Models (LLMs) succeed in many natural language processing tasks. However, their tendency to hallucinate - generate plausible but inconsistent or factually incorrect text - can cause significant problems in certain tasks, including response generation in dialogue. To mitigate this issue, we propose two novel graph knowledge-augmented frameworks, Dialogue Response Generation via Textualised Graphs (TG-DRG) and Graph-Aware Dialogue Response Generation (GA-DRG), which combine reasoning-guided dialogue reformulation, dialogue sense knowledge selection, and graph-enhanced response generation to improve the factuality of dialogue responses. To evaluate the factuality of generated responses, we propose a dialogue fact score that addresses the limitations of existing fact-score methods in dialogue settings, providing a more reliable assessment of factual consistency. We evaluate our methods using different baselines on the OpendialKG and HybriDialogue datasets. Our methods noticeably improve factuality compared to other graph knowledge-augmentation baselines, including the state-of-the-art G-retriever, achieving improvements of 3.47% on OpendialKG and 3.12% on HybriDialogue in terms of dialogue fact score. The code will be released on GitHub.