R2D2: Robust Data-to-Text with Replacement Detection

TL;DR

R2D2 is a novel training framework for Data-to-Text systems that enhances factual accuracy by combining generation and faithfulness discrimination, employing replacement detection and unlikelihood learning, leading to state-of-the-art results.

Contribution

The paper introduces R2D2, a new training approach that improves factual fidelity in Data-to-Text generation by integrating discriminator training and novel sampling methods.

Findings

R2D2 effectively reduces unfaithful text generation.

Achieves state-of-the-art results on multiple datasets.

Proposes NER-based metrics for better fidelity evaluation.

Abstract

Unfaithful text generation is a common problem for text generation systems. In the case of Data-to-Text (D2T) systems, the factuality of the generated text is particularly crucial for any real-world applications. We introduce R2D2, a training framework that addresses unfaithful Data-to-Text generation by training a system both as a generator and a faithfulness discriminator with additional replacement detection and unlikelihood learning tasks. To facilitate such training, we propose two methods for sampling unfaithful sentences. We argue that the poor entity retrieval capability of D2T systems is one of the primary sources of unfaithfulness, so in addition to the existing metrics, we further propose NER-based metrics to evaluate the fidelity of D2T generations. Our experimental results show that R2D2 systems could effectively mitigate the unfaithful text generation, and they achieve new state-of-the-art results on FeTaQA, LogicNLG, and ToTTo, all with significant improvements.