A dataset of questions on decision-theoretic reasoning in Newcomb-like problems

TL;DR

This paper introduces a dataset of natural-language questions on Newcomb-like decision problems to evaluate and analyze the reasoning and attitudes of large language models, revealing significant attitude variation and correlations with capabilities.

Contribution

It provides a novel dataset for assessing LLM reasoning on complex decision-theoretic problems and investigates model attitudes and their relationship with capabilities.

Findings

Attitudes vary significantly between models.

High capabilities correlate with favorability towards evidential decision theory.

Attitudes are consistent across different question types.

Abstract

We introduce a dataset of natural-language questions in the decision theory of so-called Newcomb-like problems. Newcomb-like problems include, for instance, decision problems in which an agent interacts with a similar other agent, and thus has to reason about the fact that the other agent will likely reason in similar ways. Evaluating LLM reasoning about Newcomb-like problems is important because interactions between foundation-model-based agents will often be Newcomb-like. Some ways of reasoning about Newcomb-like problems may allow for greater cooperation between models. Our dataset contains both capabilities questions (i.e., questions with a unique, uncontroversially correct answer) and attitude questions (i.e., questions about which decision theorists would disagree). We use our dataset for an investigation of decision-theoretical capabilities and expressed attitudes and their interplay in existing models (different models by OpenAI, Anthropic, Meta, GDM, Reka, etc.), as well as models under simple prompt-based interventions. We find, among other things, that attitudes vary significantly between existing models; that high capabilities are associated with attitudes more favorable toward so-called evidential decision theory; and that attitudes are consistent across different types of questions.