Converge to the Truth: Factual Error Correction via Iterative Constrained Editing

TL;DR

VENCE is a novel iterative editing method that corrects factual errors in claims with minimal changes by leveraging truthfulness scores and distantly-supervised models, outperforming previous methods on a public dataset.

Contribution

This paper introduces VENCE, a new approach for factual error correction that uses iterative sampling guided by truthfulness scores, requiring no extensive supervised data.

Findings

VENCE improves the SARI score by 5.3 points over previous methods.

VENCE achieves an 11.8% relative improvement on a public dataset.

The method effectively corrects multi-token factual errors with minimal edits.

Abstract

Given a possibly false claim sentence, how can we automatically correct it with minimal editing? Existing methods either require a large number of pairs of false and corrected claims for supervised training or do not handle well errors spanning over multiple tokens within an utterance. In this paper, we propose VENCE, a novel method for factual error correction (FEC) with minimal edits. VENCE formulates the FEC problem as iterative sampling editing actions with respect to a target density function. We carefully design the target function with predicted truthfulness scores from an offline trained fact verification model. VENCE samples the most probable editing positions based on back-calculated gradients of the truthfulness score concerning input tokens and the editing actions using a distantly-supervised language model (T5). Experiments on a public dataset show that VENCE improves the well-adopted SARI metric by 5.3 (or a relative improvement of 11.8%) over the previous best distantly-supervised methods.