Generating Fluent Fact Checking Explanations with Unsupervised Post-Editing

TL;DR

This paper introduces an unsupervised post-editing method that improves the fluency and coherence of fact-checking explanations generated from journalist comments, enhancing their readability and informativeness.

Contribution

It proposes a novel iterative phrase-level editing algorithm for unsupervised enhancement of fact-checking explanations, addressing fluency and coherence issues in prior extractive methods.

Findings

Generated explanations are fluent and readable.

The approach covers important information effectively.

Works well on LIAR-PLUS and PubHealth datasets.

Abstract

Fact-checking systems have become important tools to verify fake and misguiding news. These systems become more trustworthy when human-readable explanations accompany the veracity labels. However, manual collection of such explanations is expensive and time-consuming. Recent works frame explanation generation as extractive summarization, and propose to automatically select a sufficient subset of the most important facts from the ruling comments (RCs) of a professional journalist to obtain fact-checking explanations. However, these explanations lack fluency and sentence coherence. In this work, we present an iterative edit-based algorithm that uses only phrase-level edits to perform unsupervised post-editing of disconnected RCs. To regulate our editing algorithm, we use a scoring function with components including fluency and semantic preservation. In addition, we show the applicability of our approach in a completely unsupervised setting. We experiment with two benchmark datasets, LIAR-PLUS and PubHealth. We show that our model generates explanations that are fluent, readable, non-redundant, and cover important information for the fact check.