Neural scaling laws for an uncertain world

TL;DR

This paper explores how neural systems can optimally allocate resources across diverse environments by adhering to principles that minimize assumptions and maximize information transfer, explaining observed sensory and cognitive scaling laws.

Contribution

It introduces a general framework for neural resource distribution based on uncertainty principles, linking neural scaling laws to behavioral phenomena like Weber-Fechner law.

Findings

Resembles the structure of the visual system

Provides explanation for Weber-Fechner law

Suggests similar scaling in cognitive representations

Abstract

Autonomous neural systems must efficiently process information in a wide range of novel environments, which may have very different statistical properties. We consider the problem of how to optimally distribute receptors along a one-dimensional continuum consistent with the following design principles. First, neural representations of the world should obey a neural uncertainty principle---making as few assumptions as possible about the statistical structure of the world. Second, neural representations should convey, as much as possible, equivalent information about environments with different statistics. The results of these arguments resemble the structure of the visual system and provide a natural explanation of the behavioral Weber-Fechner law, a foundational result in psychology. Because the derivation is extremely general, this suggests that similar scaling relationships should be observed not only in sensory continua, but also in neural representations of ``cognitive' one-dimensional quantities such as time or numerosity.