Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge

TL;DR

This paper investigates how finetuning enables transformers to perform in-context reasoning and contextual recall, revealing that finetuning induces low-dimensional representations necessary for this ability, which pretraining alone does not produce.

Contribution

It introduces a synthetic framework to analyze contextual recall, demonstrating that finetuning on related tasks induces the necessary representations and mechanisms for in-context reasoning.

Findings

Pretraining yields factual knowledge but not contextual recall.

Finetuning on specific tasks triggers emergence of contextual recall.

Low-dimensional latent encodings of attribute types form after finetuning.

Abstract

Transformer-based language models excel at in-context learning (ICL), where they can adapt to new tasks based on contextual examples, without parameter updates. In a specific form of ICL, which we refer to as \textit{contextual recall}, models pretrained on open-ended text leverage pairwise examples to recall specific facts in novel prompt formats. We investigate whether contextual recall emerges from pretraining alone, what finetuning is required, and what mechanisms drive the necessary representations. For this, we introduce a controlled synthetic framework where pretraining sequences consist of subject-grammar-attribute tuples, with attribute types tied to grammar statistics. We demonstrate that while such pretraining successfully yields factual knowledge, it is insufficient for contextual recall: models fail to implicitly infer attribute types when the grammar statistics are removed in ICL prompts. However, we show that finetuning on tasks requiring implicit inference, distinct from the ICL evaluation, using a subset of subjects, triggers the emergence of contextual recall across all subjects. This transition is accompanied by the formation of low-dimensional latent encodings of the shared attribute type. For mechanistic insight, we derive a construction for an attention-only transformer that replicates the transition from factual to contextual recall, corroborated by empirical validation.