PARENTing via Model-Agnostic Reinforcement Learning to Correct Pathological Behaviors in Data-to-Text Generation

TL;DR

This paper introduces a model-agnostic reinforcement learning framework using the PARENT metric to effectively reduce hallucinations and omissions in data-to-text generation models, improving their factual accuracy.

Contribution

It presents a novel reinforcement learning approach that is model-agnostic and leverages the PARENT metric to correct pathological behaviors in data-to-text generation.

Findings

Significant reduction in hallucinations and omissions.

Improved performance on WikiBIO and WebNLG benchmarks.

Outperforms state-of-the-art models in factual accuracy.

Abstract

In language generation models conditioned by structured data, the classical training via maximum likelihood almost always leads models to pick up on dataset divergence (i.e., hallucinations or omissions), and to incorporate them erroneously in their own generations at inference. In this work, we build ontop of previous Reinforcement Learning based approaches and show that a model-agnostic framework relying on the recently introduced PARENT metric is efficient at reducing both hallucinations and omissions. Evaluations on the widely used WikiBIO and WebNLG benchmarks demonstrate the effectiveness of this framework compared to state-of-the-art models.