Language Agents Mirror Human Causal Reasoning Biases. How Can We Help Them Think Like Scientists?

TL;DR

This paper investigates whether language model agents can effectively explore and infer causal relationships, revealing biases similar to humans and proposing a sampling method to improve their scientific reasoning capabilities.

Contribution

The study demonstrates that LMs exhibit a disjunctive reasoning bias and introduces a test-time sampling method to mitigate this bias, enhancing causal inference.

Findings

LMS reliably infer disjunctive causal relationships

LMS struggle with conjunctive causal relationships

Proposed sampling method reduces disjunctive bias

Abstract

Language model (LM) agents are increasingly used as autonomous decision-makers which need to actively gather information to guide their decisions. A crucial cognitive skill for such agents is the efficient exploration and understanding of the causal structure of the world -- key to robust, scientifically grounded reasoning. Yet, it remains unclear whether LMs possess this capability or exhibit systematic biases leading to erroneous conclusions. In this work, we examine LMs' ability to explore and infer causal relationships, using the well-established Blicket Test paradigm from developmental psychology. We find that LMs reliably infer the common, intuitive disjunctive causal relationships but systematically struggle with the unusual, yet equally (or sometimes even more) evidenced conjunctive ones. This "disjunctive bias" persists across model families, sizes, and prompting strategies, and performance further declines as task complexity increases. Interestingly, an analogous bias appears in human adults, suggesting that LMs may have inherited deep-seated reasoning heuristics from their training data. To this end, we quantify similarities between LMs and humans, finding that LMs exhibit adult-like inference profiles (but not child-like). Finally, we propose a test-time sampling method which explicitly samples and eliminates hypotheses about causal relationships from the LM. This scalable approach significantly reduces the disjunctive bias and moves LMs closer to the goal of scientific, causally rigorous reasoning.