Scale-Invariance Drives Convergence in AI and Brain Representations

TL;DR

This paper demonstrates that scale-invariance is a key principle driving the convergence of AI and brain representations, with larger datasets and multimodal training enhancing this property and improving neural alignment.

Contribution

It introduces a multi-scale analytical framework to quantify scale-invariance in AI models and links this property to better alignment with neural data, revealing a fundamental structural principle.

Findings

Embeddings with higher scale-invariance align better with fMRI data.

Larger datasets and multimodal training improve scale-invariance and neural alignment.

fMRI manifold structures are more concentrated at smaller scales.

Abstract

Despite variations in architecture and pretraining strategies, recent studies indicate that large-scale AI models often converge toward similar internal representations that also align with neural activity. We propose that scale-invariance, a fundamental structural principle in natural systems, is a key driver of this convergence. In this work, we propose a multi-scale analytical framework to quantify two core aspects of scale-invariance in AI representations: dimensional stability and structural similarity across scales. We further investigate whether these properties can predict alignment performance with functional Magnetic Resonance Imaging (fMRI) responses in the visual cortex. Our analysis reveals that embeddings with more consistent dimension and higher structural similarity across scales align better with fMRI data. Furthermore, we find that the manifold structure of fMRI data is more concentrated, with most features dissipating at smaller scales. Embeddings with similar scale patterns align more closely with fMRI data. We also show that larger pretraining datasets and the inclusion of language modalities enhance the scale-invariance properties of embeddings, further improving neural alignment. Our findings indicate that scale-invariance is a fundamental structural principle that bridges artificial and biological representations, providing a new framework for evaluating the structural quality of human-like AI systems.