The Role of Bio-Inspired Modularity in General Learning

TL;DR

This paper explores how bio-inspired modular network architectures can enhance general learning by preventing catastrophic forgetting and enabling efficient bootstrapping, inspired by features observed in biological brains.

Contribution

It proposes that modularity in network topology, inspired by biological brains, can improve continual learning and bootstrapping without overwriting prior knowledge.

Findings

Modular network topology helps prevent catastrophic forgetting.

Biological brain features like modularity can be integrated into artificial systems.

Modularity facilitates faster learning of new tasks by leveraging prior knowledge.

Abstract

One goal of general intelligence is to learn novel information without overwriting prior learning. The utility of learning without forgetting (CF) is twofold: first, the system can return to previously learned tasks after learning something new. In addition, bootstrapping previous knowledge may allow for faster learning of a novel task. Previous approaches to CF and bootstrapping are primarily based on modifying learning in the form of changing weights to tune the model to the current task, overwriting previously tuned weights from previous tasks.However, another critical factor that has been largely overlooked is the initial network topology, or architecture. Here, we argue that the topology of biological brains likely evolved certain features that are designed to achieve this kind of informational conservation. In particular, we consider that the highly conserved property of modularity may offer a solution to weight-update learning methods that adheres to the learning without catastrophic forgetting and bootstrapping constraints. Final considerations are then made on how to combine these two learning objectives in a dynamical, general learning system.