Skewed Neuronal Heterogeneity Enhances Efficiency On Various Computing Systems

TL;DR

This paper demonstrates that inherent skewed neuronal heterogeneity improves efficiency and robustness in various computing systems, reducing network size and energy costs, inspired by biological neural variability.

Contribution

It reveals that intrinsic skewed heterogeneity enhances performance across tasks, enabling smaller, more energy-efficient neural networks and neuromorphic systems.

Findings

Skewed heterogeneity profiles boost task performance and robustness.

Intrinsic heterogeneity reduces network size and energy consumption.

Biological heterogeneity inspires efficient artificial system design.

Abstract

Heterogeneity is a ubiquitous property of many biological systems and has profound implications for computation. While it is conceivable to optimize neuronal and synaptic heterogeneity for a specific task, such top-down optimization is biologically implausible, prone to catastrophic forgetting, and both data- and energy-intensive. In contrast, biological organisms, with remarkable capacity to perform numerous tasks with minimal metabolic cost, exhibit a heterogeneity that is inherent, stable during adulthood, and task-unspecific. Inspired by this intrinsic form of heterogeneity, we investigate the utility of variations in neuronal time constants for solving hundreds of distinct temporal tasks of varying complexity. Our results show that intrinsic heterogeneity significantly enhances performance and robustness in an implementation-independent manner, indicating its usefulness for both (rate-based) machine learning and (spike-coded) neuromorphic applications. Importantly, only skewed heterogeneity profiles-reminiscent of those found in biology-produce such performance gains. We further demonstrate that this computational advantage eliminates the need for large networks, allowing comparable performance with substantially lower operational, metabolic, and energetic costs, respectively in silico, in vivo, and on neuromorphic hardware. Finally, we discuss the implications of intrinsic (rather than task-induced) heterogeneity for the design of efficient artificial systems, particularly novel neuromorphic devices that exhibit similar device-to-device variability.