The Convergence Gap: Instruction-Tuned Language Models Stabilize Later in the Forward Pass

TL;DR

This paper introduces the convergence gap diagnostic to analyze how instruction-tuned language models stabilize later in their forward pass, revealing late-stage MLP contributions as key to this delay.

Contribution

It uncovers that instruction-tuned models settle predictions later than pretrained models and identifies late MLP layers as critical leverage points for this behavior.

Findings

Instruction-tuned checkpoints remain farther from final predictions later in the stack.

Late MLP layers significantly influence the convergence delay.

Matched-prefix interventions show late MLP modifications alter prediction dynamics.

Abstract

Final outputs hide when a checkpoint commits to its next-token prediction. We introduce the convergence gap, a model-diffing diagnostic that decodes each layer's next-token distribution and measures its distance to the model's own final distribution. Across six paired pretrained and instruction-tuned checkpoints in native prompting regimes, instruction-tuned checkpoints remain farther from their final predictions later into the stack. The effect persists under endpoint-matched raw and tuned readouts, endpoint-free same-history checks, and fixed-history template replay. Matched-prefix interventions identify late MLP windows as the largest tested leverage point: late IT grafts into PT hosts increase late KL by +0.34 nats, while PT-late swaps into IT hosts reduce it by -0.51 nats; matched random late perturbations give only +0.003 versus +0.327 for the true late graft. A preselected Gemma case study provides behavior-facing plausibility for the same late swap, without serving as a benchmark claim. These results identify a robust predictiondynamics signature of post-training: released instruction-following checkpoints tend to settle later, and late MLP computation is the strongest tested bidirectional handle on that delay under matched histories.