Scalable Fact-checking with Human-in-the-Loop

TL;DR

This paper presents a scalable fact-checking approach that groups and summarizes social media posts to reduce redundancy and accelerate verification, combining automated clustering with human evaluation.

Contribution

It introduces a method to organize and summarize large volumes of social media data for fact-checking, integrating semantic graph clustering and claim summarization.

Findings

Reduced 28,818 messages to 700 summaries

Achieved high ROUGE scores for summaries

Demonstrated potential to speed up fact-checking processes

Abstract

Researchers have been investigating automated solutions for fact-checking in a variety of fronts. However, current approaches often overlook the fact that the amount of information released every day is escalating, and a large amount of them overlap. Intending to accelerate fact-checking, we bridge this gap by grouping similar messages and summarizing them into aggregated claims. Specifically, we first clean a set of social media posts (e.g., tweets) and build a graph of all posts based on their semantics; Then, we perform two clustering methods to group the messages for further claim summarization. We evaluate the summaries both quantitatively with ROUGE scores and qualitatively with human evaluation. We also generate a graph of summaries to verify that there is no significant overlap among them. The results reduced 28,818 original messages to 700 summary claims, showing the potential to speed up the fact-checking process by organizing and selecting representative claims from massive disorganized and redundant messages.