Grokking as a Falsifiable Finite-Size Transition

TL;DR

This paper rigorously tests the hypothesis that grokking is a finite-size phase transition by applying condensed matter diagnostics, providing a falsifiable framework that moves beyond analogy-based descriptions.

Contribution

It introduces a finite-size analysis framework for grokking, using spectral contrast and Binder-like crossings to quantitatively assess phase transition behavior.

Findings

Finite-size boundary identified via Binder-like crossings

Susceptibility analysis disfavors smooth crossover interpretation

Supports phase transition characterization of grokking

Abstract

Grokking -- the delayed onset of generalization after early memorization -- is often described with phase-transition language, but that claim has lacked falsifiable finite-size inputs. Here we supply those inputs by treating the group order $p$ of $\mathbb{Z}_p$ as an admissible extensive variable and a held-out spectral head-tail contrast as a representation-level order parameter, then apply a condensed-matter-style diagnostic chain to coarse-grid sweeps and a dense near-critical addition audit. Binder-like crossings reveal a shared finite-size boundary, and susceptibility comparison strongly disfavors a smooth-crossover interpretation ($\Delta\mathrm{AIC}=16.8$ in the near-critical audit). Phase-transition language in grokking can therefore be tested as a quantitative finite-size claim rather than invoked as analogy alone, although the transition order remains unresolved at present.