FLEEK: Factual Error Detection and Correction with Evidence Retrieved from External Knowledge

TL;DR

FLEEK is an automated tool that detects and corrects factual errors in text by extracting claims, retrieving external evidence, and suggesting revisions, demonstrating promising initial results.

Contribution

The paper introduces FLEEK, a novel system that automates factual error detection and correction by integrating claim extraction, external evidence retrieval, and revision suggestions.

Findings

Achieved 77-85% F1 in factual error detection

Demonstrated effective evidence retrieval for claims

Showcased potential for reducing manual fact-checking

Abstract

Detecting factual errors in textual information, whether generated by large language models (LLM) or curated by humans, is crucial for making informed decisions. LLMs' inability to attribute their claims to external knowledge and their tendency to hallucinate makes it difficult to rely on their responses. Humans, too, are prone to factual errors in their writing. Since manual detection and correction of factual errors is labor-intensive, developing an automatic approach can greatly reduce human effort. We present FLEEK, a prototype tool that automatically extracts factual claims from text, gathers evidence from external knowledge sources, evaluates the factuality of each claim, and suggests revisions for identified errors using the collected evidence. Initial empirical evaluation on fact error detection (77-85\% F1) shows the potential of FLEEK. A video demo of FLEEK can be found at https://youtu.be/NapJFUlkPdQ.