Evidence-based Factual Error Correction

TL;DR

This paper proposes a new task called factual error correction, which involves editing claims to better align with evidence, extending fact verification with a method that learns corrections from existing datasets.

Contribution

It introduces a novel two-stage distant supervision approach using T5 transformer and retrieved evidence for factual error correction, outperforming previous methods.

Findings

Achieved better correction accuracy than existing models.

Produced accurate corrections for 5x more instances in human evaluation.

Increased SARI score by 0.125 over prior approaches.

Abstract

This paper introduces the task of factual error correction: performing edits to a claim so that the generated rewrite is better supported by evidence. This extends the well-studied task of fact verification by providing a mechanism to correct written texts that are refuted or only partially supported by evidence. We demonstrate that it is feasible to train factual error correction systems from existing fact checking datasets which only contain labeled claims accompanied by evidence, but not the correction. We achieve this by employing a two-stage distant supervision approach that incorporates evidence into masked claims when generating corrections. Our approach, based on the T5 transformer and using retrieved evidence, achieved better results than existing work which used a pointer copy network and gold evidence, producing accurate factual error corrections for 5x more instances in human evaluation and a .125 increase in SARI score. The evaluation is conducted on a dataset of 65,000 instances based on a recent fact verification shared task and we release it to enable further work on the task.