Brain development dictates energy constraints on neural architecture search: cross-disciplinary insights on optimization strategies

TL;DR

This paper proposes that energy constraints, rather than prediction error minimization, primarily guide neural architecture development, offering new insights for AI NAS strategies inspired by neuroscience.

Contribution

It introduces a metabolically-focused perspective on neural architecture search, grounded in developmental neuroscience and the dynamic coordination theory, challenging traditional prediction-error-based methods.

Findings

Neural development is driven by energy efficiency constraints.

Glial-neural organization supports metabolically optimized architectures.

Implications for AI NAS and causal reasoning in neural networks.

Abstract

Present day artificial neural architecture search (NAS) strategies are essentially prediction-error-optimized. That holds true for AI functions in general. From the developmental neuroscience perspective, I present evidence for the central role of metabolically, rather than prediction-error-optimized neural architecture search (NAS). Supporting evidence is drawn from the latest insights into the glial-neural organization of the human brain and the dynamic coordination theory which provides a mathematical foundation for the functional expression of this optimization strategy. This is relevant to devising novel NAS strategies in AI, especially in AGI. Additional implications arise for causal reasoning from deep neural nets. Together, the insights from developmental neuroscience offer a new perspective on NAS and the foundational assumptions in AI modeling.