Modular Boundaries in Recurrent Neural Networks

TL;DR

This paper explores whether modular boundaries in recurrent neural networks are meaningful by applying community detection techniques to identify neural modules and examining their significance in network dynamics.

Contribution

It introduces a method to detect neural modules in RNNs using community detection and assesses their importance for network function.

Findings

Modular boundaries can be identified in RNNs using community detection.

These boundaries influence the network's dynamical properties.

Modular structure correlates with task-specific neural activity.

Abstract

Recent theoretical and experimental work in neuroscience has focused on the representational and dynamical character of neural manifolds --subspaces in neural activity space wherein many neurons coactivate. Importantly, neural populations studied under this "neural manifold hypothesis" are continuous and not cleanly divided into separate neural populations. This perspective clashes with the "modular hypothesis" of brain organization, wherein neural elements maintain an "all-or-nothing" affiliation with modules. In line with this modular hypothesis, recent research on recurrent neural networks suggests that multi-task networks become modular across training, such that different modules specialize for task-general dynamical motifs. If the modular hypothesis is true, then it would be important to use a dimensionality reduction technique that captures modular structure. Here, we investigate the features of such a method. We leverage RNNs as a model system to study the character of modular neural populations, using a community detection method from network science known as modularity maximization to partition neurons into distinct modules. These partitions allow us to ask the following question: do these modular boundaries matter to the system? ...