Large Language Models for Constrained-Based Causal Discovery

TL;DR

This paper investigates using Large Language Models as an alternative to domain experts for causal graph construction, employing prompts for conditional independence queries and enhancing performance with a voting schema.

Contribution

It introduces a novel LLM-based approach for causal discovery, framing independence queries as prompts and improving accuracy with a voting mechanism.

Findings

LLMs can perform causal reasoning with variable accuracy.

A voting schema improves false-positive and false-negative control.

Evidence suggests LLMs could complement data-driven causal discovery methods.

Abstract

Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. Constructing causal graphs often relies on either data-driven or expert-driven approaches, both fraught with challenges. The former methods, like the celebrated PC algorithm, face issues with data requirements and assumptions of causal sufficiency, while the latter demand substantial time and domain knowledge. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation. We frame conditional independence queries as prompts to LLMs and employ the PC algorithm with the answers. The performance of the LLM-based conditional independence oracle on systems with known causal graphs shows a high degree of variability. We improve the performance through a proposed statistical-inspired voting schema that allows some control over false-positive and false-negative rates. Inspecting the chain-of-thought argumentation, we find causal reasoning to justify its answer to a probabilistic query. We show evidence that knowledge-based CIT could eventually become a complementary tool for data-driven causal discovery.