When retrieval outperforms generation: Dense evidence retrieval for scalable fake news detection

TL;DR

This paper introduces DeReC, a dense retrieval-based fact verification system that outperforms LLM-based methods in accuracy and efficiency, making fake news detection more practical for real-world use.

Contribution

DeReC demonstrates that general-purpose text embeddings combined with classification can replace costly LLMs in fact verification, achieving higher accuracy and efficiency.

Findings

DeReC reduces runtime by over 90% compared to LLM-based methods.

DeReC achieves an F1 score of 65.58%, surpassing the previous state-of-the-art.

DeReC is effective across datasets of varying sizes.

Abstract

The proliferation of misinformation necessitates robust yet computationally efficient fact verification systems. While current state-of-the-art approaches leverage Large Language Models (LLMs) for generating explanatory rationales, these methods face significant computational barriers and hallucination risks in real-world deployments. We present DeReC (Dense Retrieval Classification), a lightweight framework that demonstrates how general-purpose text embeddings can effectively replace autoregressive LLM-based approaches in fact verification tasks. By combining dense retrieval with specialized classification, our system achieves better accuracy while being significantly more efficient. DeReC outperforms explanation-generating LLMs in efficiency, reducing runtime by 95% on RAWFC (23 minutes 36 seconds compared to 454 minutes 12 seconds) and by 92% on LIAR-RAW (134 minutes 14 seconds compared to 1692 minutes 23 seconds), showcasing its effectiveness across varying dataset sizes. On the RAWFC dataset, DeReC achieves an F1 score of 65.58%, surpassing the state-of-the-art method L-Defense (61.20%). Our results demonstrate that carefully engineered retrieval-based systems can match or exceed LLM performance in specialized tasks while being significantly more practical for real-world deployment.