Modeling Higher-Order Interactions in Sparse and Heavy-Tailed Neural Population Activity

TL;DR

This paper develops a theoretical framework to understand how specific nonlinear neuron properties lead to sparse, heavy-tailed population activity distributions, revealing mechanisms behind neural sparsity and synchrony.

Contribution

It derives conditions and proposes a distribution class explaining how nonlinearities induce sparse, heavy-tailed neural activity patterns, linking neuron properties to population dynamics.

Findings

Neurons with threshold-like and supralinear activation produce sparse, synchronous activity.

Structured higher-order interactions with alternating signs shape population firing distributions.

Recurrent networks with these properties relate to memory capacity in Hopfield-like models.

Abstract

Neurons process sensory stimuli efficiently, showing sparse yet highly variable ensemble spiking activity involving structured higher-order interactions. Notably, while neural populations are mostly silent, they occasionally exhibit highly synchronous activity, resulting in sparse and heavy-tailed spike-count distributions. However, its mechanistic origin - specifically, what types of nonlinear properties in individual neurons induce such population-level patterns - remains unclear. In this study, we derive sufficient conditions under which the joint activity of homogeneous binary neurons generates sparse and widespread population firing rate distributions in infinitely large networks. We then propose a subclass of exponential family distributions that satisfy this condition. This class incorporates structured higher-order interactions with alternating signs and shrinking magnitudes, along with a base-measure function that offsets distributional concentration, giving rise to parameter-dependent sparsity and heavy-tailed population firing rate distributions. Analysis of recurrent neural networks that recapitulate these distributions reveals that individual neurons possess threshold-like nonlinearity followed by supralinear activation that jointly facilitates sparse and synchronous population activity. These nonlinear features resemble those in modern Hopfield networks, suggesting a connection between widespread population activity and the network's memory capacity. The theory establishes sparse and heavy-tailed distributions for binary patterns, forming a foundation for developing energy-efficient spike-based learning machines.