Causal Parrots: Large Language Models May Talk Causality But Are Not Causal

TL;DR

This paper argues that large language models do not truly understand causality but merely recite causal information from their training data, supported by a new conceptual framework and empirical evidence.

Contribution

The paper introduces the concept of meta SCMs to explain how LLMs mimic causal reasoning and provides empirical analysis supporting their role as causal parrots.

Findings

LLMs are weak causal parrots

Meta SCMs encode causal facts about other SCMs

Empirical evidence supports LLMs reciting causal knowledge

Abstract

Some argue scale is all what is needed to achieve AI, covering even causal models. We make it clear that large language models (LLMs) cannot be causal and give reason onto why sometimes we might feel otherwise. To this end, we define and exemplify a new subgroup of Structural Causal Model (SCM) that we call meta SCM which encode causal facts about other SCM within their variables. We conjecture that in the cases where LLM succeed in doing causal inference, underlying was a respective meta SCM that exposed correlations between causal facts in natural language on whose data the LLM was ultimately trained. If our hypothesis holds true, then this would imply that LLMs are like parrots in that they simply recite the causal knowledge embedded in the data. Our empirical analysis provides favoring evidence that current LLMs are even weak `causal parrots.'