CauScientist: Teaching LLMs to Respect Data for Causal Discovery

TL;DR

CauScientist introduces a collaborative framework combining large language models with statistical verification to improve causal discovery accuracy and robustness over existing data-driven methods.

Contribution

The paper presents CauScientist, a novel hybrid approach that leverages LLMs for hypothesis generation and statistical methods for validation, significantly enhancing causal discovery performance.

Findings

Up to 53.8% F1 score improvement over baselines

Recall increased from 35.0% to 100.0%

44.0% reduction in structural hamming distance on complex graphs

Abstract

Causal discovery is fundamental to scientific understanding and reliable decision-making. Existing approaches face critical limitations: purely data-driven methods suffer from statistical indistinguishability and modeling assumptions, while recent LLM-based methods either ignore statistical evidence or incorporate unverified priors that can mislead result. To this end, we propose CauScientist, a collaborative framework that synergizes LLMs as hypothesis-generating "data scientists" with probabilistic statistics as rigorous "verifiers". CauScientist employs hybrid initialization to select superior starting graphs, iteratively refines structures through LLM-proposed modifications validated by statistical criteria, and maintains error memory to guide efficient search space. Experiments demonstrate that CauScientist substantially outperforms purely data-driven baselines, achieving up to 53.8% F1 score improvement and enhancing recall from 35.0% to 100.0%. Notably, while standalone LLM performance degrades with graph complexity, CauScientist reduces structural hamming distance (SHD) by 44.0% compared to Qwen3-32B on 37-node graphs. Our project page is at https://github.com/OpenCausaLab/CauScientist.