Atomic Reasoning for Scientific Table Claim Verification

TL;DR

This paper introduces a modular atomic reasoning approach for scientific table claim verification, inspired by Cognitive Load Theory, which improves accuracy and generalization with less training data compared to existing models.

Contribution

It proposes a skill-chaining schema for atomic reasoning, creating a new benchmark, SciAtomicBench, and demonstrates superior performance over GPT-4o with minimal fine-tuning.

Findings

Outperforms GPT-4o's chain-of-thought method.

Achieves state-of-the-art results with only 350 fine-tuning examples.

Reduces cognitive load through modular reasoning components.

Abstract

Scientific texts often convey authority due to their technical language and complex data. However, this complexity can sometimes lead to the spread of misinformation. Non-experts are particularly susceptible to misleading claims based on scientific tables due to their high information density and perceived credibility. Existing table claim verification models, including state-of-the-art large language models (LLMs), often struggle with precise fine-grained reasoning, resulting in errors and a lack of precision in verifying scientific claims. Inspired by Cognitive Load Theory, we propose that enhancing a model's ability to interpret table-based claims involves reducing cognitive load by developing modular, reusable reasoning components (i.e., atomic skills). We introduce a skill-chaining schema that dynamically composes these skills to facilitate more accurate and generalizable reasoning with a reduced cognitive load. To evaluate this, we create SciAtomicBench, a cross-domain benchmark with fine-grained reasoning annotations. With only 350 fine-tuning examples, our model trained by atomic reasoning outperforms GPT-4o's chain-of-thought method, achieving state-of-the-art results with far less training data.