A probabilistic study of neural complexity

TL;DR

This paper investigates the mathematical properties of neural complexity, classifies all such measures satisfying certain properties, and analyzes their behavior as system size increases, revealing symmetry-breaking phenomena.

Contribution

It introduces the concept of intricacies satisfying exchangeability and additivity, classifies them via probability laws, and studies their growth and maximizers in large systems.

Findings

Maximizers have small support.

Exchangeable systems have small intricacy.

Maximizing intricacy causes symmetry breaking.

Abstract

G. Edelman, O. Sporns, and G. Tononi have introduced the neural complexity of a family of random variables, defining it as a specific average of mutual information over subfamilies. We show that their choice of weights satisfies two natural properties, namely exchangeability and additivity, and we call any functional satisfying these two properties an intricacy. We classify all intricacies in terms of probability laws on the unit interval and study the growth rate of maximal intricacies when the size of the system goes to infinity. For systems of a fixed size, we show that maximizers have small support and exchangeable systems have small intricacy. In particular, maximizing intricacy leads to spontaneous symmetry breaking and failure of uniqueness.