Zero-shot Faithful Factual Error Correction
Kung-Hsiang Huang, Hou Pong Chan, Heng Ji

TL;DR
This paper introduces a zero-shot method for correcting factual errors in text that outperforms supervised models and offers interpretability, evaluated on FEVER and SciFact datasets.
Contribution
The paper proposes a novel zero-shot framework for factual error correction that is more faithful and interpretable than existing supervised approaches.
Findings
Outperforms supervised methods on FEVER and SciFact datasets.
Provides interpretable corrections through a decomposable framework.
Analyzes metric correlations with human judgments for evaluation.
Abstract
Faithfully correcting factual errors is critical for maintaining the integrity of textual knowledge bases and preventing hallucinations in sequence-to-sequence models. Drawing on humans' ability to identify and correct factual errors, we present a zero-shot framework that formulates questions about input claims, looks for correct answers in the given evidence, and assesses the faithfulness of each correction based on its consistency with the evidence. Our zero-shot framework outperforms fully-supervised approaches, as demonstrated by experiments on the FEVER and SciFact datasets, where our outputs are shown to be more faithful. More importantly, the decomposability nature of our framework inherently provides interpretability. Additionally, to reveal the most suitable metrics for evaluating factual error corrections, we analyze the correlation between commonly used metrics with human…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Explainable Artificial Intelligence (XAI) · Biomedical Text Mining and Ontologies
