Less is More: some Computational Principles based on Parcimony, and Limitations of Natural Intelligence

TL;DR

This paper explores how the brain's natural constraints foster efficiency, adaptability, and creativity, proposing that AI can benefit from principles of parsimony, energy limits, and embodied interaction to develop more robust and interpretable systems.

Contribution

It introduces computational principles based on neural and energy constraints, highlighting their role in natural intelligence and proposing their adoption in artificial systems for improved efficiency and interpretability.

Findings

Neural bandwidth constraints promote concise, effective coding.

Hierarchical and symbolic structures emerge naturally from bandwidth limits.

Energy constraints and active interaction can enhance AI robustness and interpretability.

Abstract

Natural intelligence (NI) consistently achieves more with less. Infants learn language, develop abstract concepts, and acquire sensorimotor skills from sparse data, all within tight neural and energy limits. In contrast, today's AI relies on virtually unlimited computational power, energy, and data to reach high performance. This paper argues that constraints in NI are paradoxically catalysts for efficiency, adaptability, and creativity. We first show how limited neural bandwidth promotes concise codes that still capture complex patterns. Spiking neurons, hierarchical structures, and symbolic-like representations emerge naturally from bandwidth constraints, enabling robust generalization. Next, we discuss chaotic itinerancy, illustrating how the brain transits among transient attractors to flexibly retrieve memories and manage uncertainty. We then highlight reservoir computing, where random projections facilitate rapid generalization from small datasets. Drawing on developmental perspectives, we emphasize how intrinsic motivation, along with responsive social environments, drives infant language learning and discovery of meaning. Such active, embodied processes are largely absent in current AI. Finally, we suggest that adopting 'less is more' principles -- energy constraints, parsimonious architectures, and real-world interaction -- can foster the emergence of more efficient, interpretable, and biologically grounded artificial systems.