Combinatorial Geometry of Threshold-Linear Networks

TL;DR

This paper introduces mathematical techniques using hyperplane arrangements and oriented matroids to analyze the dynamic regimes and fixed points of threshold-linear neural networks, revealing how architecture constrains neural dynamics.

Contribution

It develops a novel framework connecting network architecture to combinatorial properties, enabling characterization of dynamic regimes and fixed points in TLNs using oriented matroids.

Findings

Complete characterization of fixed points for networks with n=3.

Method to modulate synaptic strengths to access different dynamics.

Framework applicable to studying neural motifs and their computational roles.

Abstract

The architecture of a neural network constrains the potential dynamics that can emerge. Some architectures may only allow for a single dynamic regime, while others display a great deal of flexibility with qualitatively different dynamics that can be reached by modulating connection strengths. In this work, we develop novel mathematical techniques to study the dynamic constraints imposed by different network architectures in the context of competitive threshold-linear networks (TLNs). Any given TLN is naturally characterized by a hyperplane arrangement in $\mathbb{R}^n$, and the combinatorial properties of this arrangement determine the pattern of fixed points of the dynamics. This observation enables us to recast the question of network flexibility in the language of oriented matroids, allowing us to employ tools and results from this theory in order to characterize the different dynamic regimes a given architecture can support. In particular, fixed points of a TLN correspond to cocircuits of an associated oriented matroid; and mutations of the matroid correspond to bifurcations in the collection of fixed points. As an application, we provide a complete characterization of all possible sets of fixed points that can arise in networks through size $n=3$, together with descriptions of how to modulate synaptic strengths of the network in order to access the different dynamic regimes. These results provide a framework for studying the possible computational roles of various motifs observed in real neural networks.