LLMs Are Prone to Fallacies in Causal Inference

TL;DR

This paper investigates whether large language models can infer causal relations from relational data in text, revealing their susceptibility to fallacies and limitations in understanding causality beyond memorized facts.

Contribution

The study demonstrates that LLMs often infer causal relations based on textual order and temporal cues, highlighting their fallibility and challenges in understanding complex causal concepts.

Findings

LLMs infer causality from mention order in text.

LLMs suffer from post hoc fallacy with randomized order.

Difficulty in inferring causality from counterfactuals.

Abstract

Recent work shows that causal facts can be effectively extracted from LLMs through prompting, facilitating the creation of causal graphs for causal inference tasks. However, it is unclear if this success is limited to explicitly-mentioned causal facts in the pretraining data which the model can memorize. Thus, this work investigates: Can LLMs infer causal relations from other relational data in text? To disentangle the role of memorized causal facts vs inferred causal relations, we finetune LLMs on synthetic data containing temporal, spatial and counterfactual relations, and measure whether the LLM can then infer causal relations. We find that: (a) LLMs are susceptible to inferring causal relations from the order of two entity mentions in text (e.g. X mentioned before Y implies X causes Y); (b) if the order is randomized, LLMs still suffer from the post hoc fallacy, i.e. X occurs before Y (temporal relation) implies X causes Y. We also find that while LLMs can correctly deduce the absence of causal relations from temporal and spatial relations, they have difficulty inferring causal relations from counterfactuals, questioning their understanding of causality.