Leveraging Large Language Models for Causal Discovery: a Constraint-based, Argumentation-driven Approach

TL;DR

This paper introduces a novel approach that combines large language models with causal assumption-based argumentation to improve causal discovery, achieving state-of-the-art results by integrating semantic priors and independence evidence.

Contribution

It presents a new method leveraging LLMs as imperfect experts within a symbolic reasoning framework for causal discovery, enhancing existing techniques with semantic priors.

Findings

Achieved state-of-the-art performance on benchmark datasets.

Effectively integrated semantic priors from LLMs with independence evidence.

Proposed an evaluation protocol to reduce memorisation bias in LLM-based causal discovery.

Abstract

Causal discovery seeks to uncover causal relations from data, typically represented as causal graphs, and is essential for predicting the effects of interventions. While expert knowledge is required to construct principled causal graphs, many statistical methods have been proposed to leverage observational data with varying formal guarantees. Causal Assumption-based Argumentation (ABA) is a framework that uses symbolic reasoning to ensure correspondence between input constraints and output graphs, while offering a principled way to combine data and expertise. We explore the use of large language models (LLMs) as imperfect experts for Causal ABA, eliciting semantic structural priors from variable names and descriptions and integrating them with conditional-independence evidence. Experiments on standard benchmarks and semantically grounded synthetic graphs demonstrate state-of-the-art performance, and we additionally introduce an evaluation protocol to mitigate memorisation bias when assessing LLMs for causal discovery.