Causality Elicitation from Large Language Models

TL;DR

This paper introduces a pipeline to extract and analyze potential causal relationships from large language models by sampling documents, identifying events, grouping them, and applying causal discovery methods, providing a framework for hypothesis generation.

Contribution

The paper presents a novel framework for eliciting and visualizing plausible causal hypotheses from LLMs without asserting real-world causality.

Findings

Framework generates interpretable causal hypotheses

Method identifies common event patterns across documents

Provides a set of candidate causal graphs for analysis

Abstract

Large language models (LLMs) are trained on enormous amounts of data and encode knowledge in their parameters. We propose a pipeline to elicit causal relationships from LLMs. Specifically, (i) we sample many documents from LLMs on a given topic, (ii) we extract an event list from from each document, (iii) we group events that appear across documents into canonical events, (iv) we construct a binary indicator vector for each document over canonical events, and (v) we estimate candidate causal graphs using causal discovery methods. Our approach does not guarantee real-world causality. Rather, it provides a framework for presenting the set of causal hypotheses that LLMs can plausibly assume, as an inspectable set of variables and candidate graphs.