Tailored ensembles of neural networks optimize sensitivity to stimulus statistics

TL;DR

This paper proposes a method to tailor ensembles of neural networks to optimize their sensitivity across stimulus intensities, enabling adaptable dynamic ranges for improved stimulus discrimination.

Contribution

It introduces a framework for customizing neural network ensembles to extend and adapt their dynamic range based on stimulus distribution, overcoming coalescence limitations.

Findings

Maximal dynamic range occurs at criticality but with similar discriminable intervals across tunings.

Compensating for coalescence allows adaptation of discriminable intervals.

Ensembles can be tailored to stimulus distributions to arbitrarily extend dynamic range.

Abstract

The dynamic range of stimulus processing in living organisms is much larger than a single neural network can explain. For a generic, tunable spiking network we derive that while the dynamic range is maximal at criticality, the interval of discriminable intensities is very similar for any network tuning due to coalescence. Compensating coalescence enables adaptation of discriminable intervals. Thus, we can tailor an ensemble of networks optimized to the distribution of stimulus intensities, e.g., extending the dynamic range arbitrarily. We discuss potential applications in machine learning.