Eight challenges in developing theory of intelligence

TL;DR

This paper discusses eight key challenges in developing a comprehensive theoretical framework for understanding intelligence, both biological and artificial, emphasizing the role of abstract models and mathematical beauty.

Contribution

It identifies and elaborates on eight fundamental challenges in formulating a theory of intelligence using bottom-up mechanistic modeling.

Findings

Highlights the importance of abstract models in understanding complex systems.

Proposes that many dimensions in neural systems are sloppy, with fewer stiff dimensions impacting observables.

Emphasizes the potential of this approach for advancing AI and biological intelligence theories.

Abstract

A good theory of mathematical beauty is more practical than any current observation, as new predictions of physical reality can be verified self-consistently. This belief applies to the current status of understanding deep neural networks including large language models and even the biological intelligence. Toy models provide a metaphor of physical reality, allowing mathematically formulating that reality (i.e., the so-called theory), which can be updated as more conjectures are justified or refuted. One does not need to pack all details into a model, but rather, more abstract models are constructed, as complex systems like brains or deep networks have many sloppy dimensions but much less stiff dimensions that strongly impact macroscopic observables. This kind of bottom-up mechanistic modeling is still promising in the modern era of understanding the natural or artificial intelligence. Here, we shed light on eight challenges in developing theory of intelligence following this theoretical paradigm. Theses challenges are representation learning, generalization, adversarial robustness, continual learning, causal learning, internal model of the brain, next-token prediction, and finally the mechanics of subjective experience.