On the Reliability of Large Language Models for Causal Discovery

TL;DR

This paper evaluates the capabilities and limitations of large language models in causal discovery, highlighting their dependence on training data frequency, the impact of incorrect data, and the importance of context.

Contribution

It provides a systematic analysis of how open-source LLMs perform in causal discovery tasks, revealing their strengths and weaknesses in generalization and data reliability.

Findings

LLMs excel at frequent causal relations in training data.

Their ability to generalize to rare or new causal relations is limited.

Incorrect causal data significantly reduces LLM confidence in correct relations.

Abstract

This study investigates the efficacy of Large Language Models (LLMs) in causal discovery. Using newly available open-source LLMs, OLMo and BLOOM, which provide access to their pre-training corpora, we investigate how LLMs address causal discovery through three research questions. We examine: (i) the impact of memorization for accurate causal relation prediction, (ii) the influence of incorrect causal relations in pre-training data, and (iii) the contextual nuances that influence LLMs' understanding of causal relations. Our findings indicate that while LLMs are effective in recognizing causal relations that occur frequently in pre-training data, their ability to generalize to new or rare causal relations is limited. Moreover, the presence of incorrect causal relations significantly undermines the confidence of LLMs in corresponding correct causal relations, and the contextual information critically affects the outcomes of LLMs to discern causal connections between random variables.