# Balanced state of networks of winner-take-all units

## TL;DR

This paper introduces a new brain network model using winner-take-all units that can produce realistic spiking patterns and flexible nonlinear dynamics, important for neural computation.

## Contribution

The paper proposes a novel network model using winner-take-all units that combines realistic spiking with flexible nonlinear dynamics through a mathematical theory.

## Key findings

- Networks of winner-take-all units exhibit chaotic fluctuation-driven regimes with irregular spiking similar to real cortical activity.
- The model supports multistability, stable sequence generation, and complex firing rate dynamics via chaotic spiking.
- The model can reproduce realistic spike trains and diverse nonlinear dynamics suitable for flexible computation.

## Abstract

Irregularly timed action potentials, or spikes, are pervasively observed in the brain activity of awake mammals. However, the role of this temporal irregularity in neural computation is still not well understood. In canonical network models irregular spiking emerges via balanced, fluctuating input currents, leading to collective responses that track inputs linearly. How networks characterized by irregular spiking could support flexible nonlinear dynamics needed for general-purpose computation remains under ongoing debate. Here we characterize the dynamics of networks whose elementary unit is not a single neuron but a small group of neurons, with distinct tunings, that compete at each timestep via a winner-take-all (WTA) interaction. While WTA has long been proposed as an elementary functional motif in the brain and represents a powerful computational primitive, how large networks of such units behave has received less investigation. We show that these networks, like classic excitatory-inhibitory balanced networks, exhibit a chaotic fluctuation-driven regime characterized by sustained irregular activity resembling realistic cortical spiking, which we interpret as a multidimensional balance spread over several competing neural populations with different tunings. We develop a mean-field theory for the network, which shows how irregular spiking sustained by time-varying input fluctuations can support flexible nonlinear collective dynamics. Using the theory we predict and verify network regimes in which input fluctuations alone yield multistability, stable sequence generation, or complex heterogeneous firing rate dynamics—three core dynamical primitives thought to underlie memory-dependent neural computation—via consistent Poisson-like spiking produced through chaos. Thus, networks of WTA units support a chaotic fluctuation-driven regime characterized by irregular spiking that can power complex nonlinear collective dynamics. This represents a new model of brain activity capable of simultaneously reproducing realistic spike trains and diverse nonlinear firing rate patterns well posed for flexible computation, and which can be trained or fit to data.

The irregular timing of action potentials, or “spikes,” is one of the most pervasive features of neural activity in awake mammals, observed in a wide range of brain areas and conditions. However, the role of this striking temporal irregularity in neural information processing is still not understood. In network models, it has been particularly challenging to reconcile irregular spiking with flexible nonlinear “firing rate” dynamics, which are thought to be crucial for general-purpose neural computation. Here we present a network model, based on winner-take-all units akin to small groups of neurons, in which irregular spiking and flexible nonlinear dynamics go hand-in-hand and can be related via a concise mathematical theory. The result is a new model of brain activity capable of simultaneously capturing realistic spiking and firing rate dynamics, which is well posed for flexible computation and can be trained or fit to data.