An evolutionary perspective on modes of learning in Transformers

TL;DR

This paper explores how environmental stability and cue reliability influence the preference for in-weight learning or in-context learning in Transformers, revealing task-dependent strategy transitions driven by environmental and task factors.

Contribution

It introduces an evolutionary framework to understand the conditions favoring different learning modes in Transformers, supported by controlled experiments and analysis.

Findings

Stable environments favor in-weight learning (IWL).

Reliable cues promote in-context learning (ICL) in volatile environments.

Transitions between strategies depend on environmental and task-specific factors.

Abstract

The success of Transformers lies in their ability to improve inference through two complementary strategies: the permanent refinement of model parameters via in-weight learning (IWL), and the ephemeral modulation of inferences via in-context learning (ICL), which leverages contextual information maintained in the model's activations. Evolutionary biology tells us that the predictability of the environment across timescales predicts the extent to which analogous strategies should be preferred. Genetic evolution adapts to stable environmental features by gradually modifying the genotype over generations. Conversely, environmental volatility favors plasticity, which enables a single genotype to express different traits within a lifetime, provided there are reliable cues to guide the adaptation. We operationalize these dimensions (environmental stability and cue reliability) in controlled task settings (sinusoid regression and Omniglot classification) to characterize their influence on learning in Transformers. We find that stable environments favor IWL, often exhibiting a sharp transition when conditions are static. Conversely, reliable cues favor ICL, particularly when the environment is volatile. Furthermore, an analysis of learning dynamics reveals task-dependent transitions between strategies (ICL to IWL and vice versa). We demonstrate that these transitions are governed by (1) the asymptotic optimality of the strategy with respect to the environment, and (2) the optimization cost of acquiring that strategy, which depends on the task structure and the learner's inductive bias.