# Emergence of functionally differentiated structures via mutual information minimization in recurrent neural networks

## TL;DR

This paper shows how minimizing mutual information in recurrent neural networks leads to the emergence of functionally specialized modules, similar to how the brain develops.

## Contribution

A novel method for inducing functional differentiation in recurrent neural networks by minimizing mutual information between subgroups.

## Key findings

- Mutual information minimization leads to high task performance and functional modularity in neural networks.
- Functional differentiation emerges before structural modularity in network development.
- The method was successfully applied to working memory and chaotic signal separation tasks.

## Abstract

Functional differentiation in the brain emerges as distinct regions specialize and is key to understanding brain function as a complex system. Previous research has modeled this process using artificial neural networks with specific constraints. Here, we propose a novel approach that induces functional differentiation in recurrent neural networks by minimizing mutual information between neural subgroups via mutual information neural estimation. We apply our method to a 2-bit working memory task and a chaotic signal separation task involving Lorenz and Rössler time series. Analysis of network performance, correlation patterns, and weight matrices reveals that mutual information minimization yields high task performance alongside clear functional modularity and moderate structural modularity. Importantly, our results show that functional differentiation, which is measured through correlation structures, emerges earlier than structural modularity defined by synaptic weights. This suggests that functional specialization precedes and probably drives structural reorganization within developing neural networks. Our findings provide new insights into how information-theoretic principles may govern the emergence of specialized functions and modular structures during artificial and biological brain development.