Neutral stability, rate propagation, and critical branching in feedforward networks

TL;DR

This paper investigates how simple feedforward neural networks can exhibit critical properties like stability and diverse activity patterns, focusing on how thresholds and connectivity influence these dynamics.

Contribution

It identifies the conditions under which critical branching and neutral stability emerge in random feedforward networks of McCullochs-Pitts neurons, extending analysis to include inhibition and noise.

Findings

Low thresholds enable properties like firing rate preservation and broad activity distributions.

High thresholds prevent these properties from occurring.

Inhibition and noise extend the parameter space where these properties are observed.

Abstract

Recent experimental and computational evidence suggests that several dynamical properties may characterize the operating point of functioning neural networks: critical branching, neutral stability, and production of a wide range of firing patterns. We seek the simplest setting in which these properties emerge, clarifying their origin and relationship in random, feedforward networks of McCullochs-Pitts neurons. Two key parameters are the thresholds at which neurons fire spikes, and the overall level of feedforward connectivity. When neurons have low thresholds, we show that there is always a connectivity for which the properties in question all occur: that is, these networks preserve overall firing rates from layer to layer and produce broad distributions of activity in each layer. This fails to occur, however, when neurons have high thresholds. A key tool in explaining this difference is eigenstructure of the resulting mean-field Markov chain, as this reveals which activity modes will be preserved from layer to layer. We extend our analysis from purely excitatory networks to more complex models that include inhibition and 'local' noise, and find that both of these features extend the parameter ranges over which networks produce the properties of interest.