Out-of-Context Abduction: LLMs Make Inferences About Procedural Data Leveraging Declarative Facts in Earlier Training Data

TL;DR

This paper investigates whether large language models can infer and reason about procedural data and facts from their training data, demonstrating their ability to make out-of-context abductions and implications for AI safety.

Contribution

It introduces experimental methods to assess out-of-context abduction in LLMs and shows GPT-4o's ability to infer chatbot identities and behaviors from training data.

Findings

GPT-4o infers chatbot names from responses

Training on behavior descriptions enhances chatbot behavior display

Results impact understanding of LLMs' situational awareness

Abstract

Large language models (LLMs) are trained on large corpora, yet it is unclear whether they can reason about the information present within their training data. We design experiments to study out-of-context abduction in LLMs, the ability to infer the most plausible explanations for observations using relevant facts present in training data. We train treatment LLMs on names and behavior descriptions of fictitious chatbots, but not on examples of dialogue with the chatbots. We find that OpenAI's GPT 4o LLM can correctly infer at least one chatbot's name after observing example responses characteristic of that chatbot. We also find that previously training GPT 4o on descriptions of a chatbot's behavior allows it to display behaviors more characteristic of the chatbot when iteratively trained to display such behaviors. Our results have implications for situational awareness in LLMs and, therefore, for AI safety.