Althea: Human-AI Collaboration for Fact-Checking and Critical Reasoning

TL;DR

Althea is a human-AI system designed to enhance fact-checking and reasoning by integrating retrieval, structured questioning, and scaffolding, leading to improved accuracy and confidence in evaluating online claims.

Contribution

The paper introduces Althea, a novel retrieval-augmented system that combines question generation, evidence retrieval, and scaffolding to improve human-AI collaborative fact-checking.

Findings

Guided interaction improves immediate accuracy and confidence.

Self-directed search yields long-term improvements.

Participants find Althea transparent and supportive of reasoning.

Abstract

The web's information ecosystem demands fact-checking systems that are both scalable and epistemically trustworthy. Automated approaches offer efficiency but often lack transparency, while human verification remains slow and inconsistent. We introduce Althea, a retrieval-augmented system that integrates question generation, evidence retrieval, and structured reasoning to support user-driven evaluation of online claims. On the AVeriTeC benchmark, Althea achieves a Macro-F1 of 0.44, outperforming standard verification pipelines and improving discrimination between supported and refuted claims. We further evaluate Althea through a controlled user study and a longitudinal survey experiment (N=963), comparing three interaction modes that vary in the degree of scaffolding: an Exploratory mode with guided reasoning, a Summary mode providing synthesized verdicts, and a Self-search mode that offers procedural guidance without algorithmic intervention. Results show that guided interaction produces the strongest immediate gains in accuracy and confidence, while self-directed search yields the most persistent improvements over time. This pattern suggests that performance gains are not driven solely by effort or exposure, but by how cognitive work is structured and internalized. Participants consistently described Althea as transparent and supportive of reflective reasoning, emphasizing its ability to organize evidence and clarify competing claims. By integrating retrieval, interaction, and pedagogical scaffolding, Althea demonstrates how human--AI interaction can move beyond automated verdicts toward durable improvements in reasoning. These findings advance the design of trustworthy, human-centered fact-checking systems that balance guidance with epistemic autonomy.