Transformers self-organize like newborn visual systems when trained in prenatal worlds

TL;DR

Transformers trained on simulated prenatal visual data develop structures similar to newborn brains, indicating shared learning principles between artificial models and biological development.

Contribution

This study demonstrates that transformers can spontaneously develop brain-like visual structures when trained on prenatal-like data, bridging AI and neuroscience.

Findings

Transformers develop edge sensitivity in early layers.

Later layers become shape-sensitive.

Receptive fields grow across layers during training.

Abstract

Do transformers learn like brains? A key challenge in addressing this question is that transformers and brains are trained on fundamentally different data. Brains are initially "trained" on prenatal sensory experiences (e.g., retinal waves), whereas transformers are typically trained on large datasets that are not biologically plausible. We reasoned that if transformers learn like brains, then they should develop the same structure as newborn brains when exposed to the same prenatal data. To test this prediction, we simulated prenatal visual input using a retinal wave generator. Then, using self-supervised temporal learning, we trained transformers to adapt to those retinal waves. During training, the transformers spontaneously developed the same structure as newborn visual systems: (1) early layers became sensitive to edges, (2) later layers became sensitive to shapes, and (3) the models developed larger receptive fields across layers. The organization of newborn visual systems emerges spontaneously when transformers adapt to a prenatal visual world. This developmental convergence suggests that brains and transformers learn in common ways and follow the same general fitting principles.