Prompting Strategies for Enabling Large Language Models to Infer Causation from Correlation

TL;DR

This paper presents a prompting strategy called PC-SubQ that guides large language models to perform causal reasoning by breaking down the task into steps aligned with the PC causal discovery algorithm, improving their ability to infer causation from correlation.

Contribution

The paper introduces PC-SubQ, a novel prompting method that enhances LLMs' causal inference by decomposing the task into formal algorithmic steps, demonstrating improved performance on causal benchmarks.

Findings

Performance improved across five LLMs with PC-SubQ

Robustness to variable name changes and paraphrasing

Effective in causal inference from correlation

Abstract

The reasoning abilities of Large Language Models (LLMs) are attracting increasing attention. In this work, we focus on causal reasoning and address the task of establishing causal relationships based on correlation information, a highly challenging problem on which several LLMs have shown poor performance. We introduce a prompting strategy for this problem that breaks the original task into fixed subquestions, with each subquestion corresponding to one step of a formal causal discovery algorithm, the PC algorithm. The proposed prompting strategy, PC-SubQ, guides the LLM to follow these algorithmic steps, by sequentially prompting it with one subquestion at a time, augmenting the next subquestion's prompt with the answer to the previous one(s). We evaluate our approach on an existing causal benchmark, Corr2Cause: our experiments indicate a performance improvement across five LLMs when comparing PC-SubQ to baseline prompting strategies. Results are robust to causal query perturbations, when modifying the variable names or paraphrasing the expressions.