Thousand-Brains Systems: Sensorimotor Intelligence for Rapid, Robust Learning and Inference

TL;DR

This paper introduces Monty, a prototype of a thousand-brains system inspired by cortical columns, demonstrating robust sensorimotor learning, rapid inference, and efficient continual learning for 3D object perception.

Contribution

It presents the first implementation of a thousand-brains architecture, showcasing its advantages in sensorimotor grounded representations, rapid inference, and continual learning over traditional AI models.

Findings

Monty effectively generalizes object recognition and pose estimation.

Sensorimotor representations enable robust and efficient learning.

Modular architecture with voting accelerates inference speed.

Abstract

Current AI systems achieve impressive performance on many tasks, yet they lack core attributes of biological intelligence, including rapid, continual learning, representations grounded in sensorimotor interactions, and structured knowledge that enables efficient generalization. Neuroscience theory suggests that mammals evolved flexible intelligence through the replication of a semi-independent, sensorimotor module, a functional unit known as a cortical column. To address the disparity between biological and artificial intelligence, thousand-brains systems were proposed as a means of mirroring the architecture of cortical columns and their interactions. In the current work, we evaluate the unique properties of Monty, the first implementation of a thousand-brains system. We focus on 3D object perception, and in particular, the combined task of object recognition and pose estimation. Utilizing the YCB dataset of household objects, we first assess Monty's use of sensorimotor learning to build structured representations, finding that these enable robust generalization. These representations include an emphasis on classifying objects by their global shape, as well as a natural ability to detect object symmetries. We then explore Monty's use of model-free and model-based policies to enable rapid inference by supporting principled movements. We find that such policies complement Monty's modular architecture, a design that can accommodate communication between modules to further accelerate inference speed via a novel `voting' algorithm. Finally, we examine Monty's use of associative, Hebbian-like binding to enable rapid, continual, and computationally efficient learning, properties that compare favorably to current deep learning architectures. While Monty is still in a nascent stage of development, these findings support thousand-brains systems as a powerful and promising new approach to AI.