Neuro-inspired Ensemble-to-Ensemble Communication Primitives for Sparse and Efficient ANNs

TL;DR

This paper introduces G2GNet, a biologically inspired sparse neural network architecture that uses ensemble-to-ensemble connectivity, dynamic sparse training, and Hebbian-inspired rewiring to improve accuracy and efficiency on vision benchmarks.

Contribution

It presents the first ANN architecture based on biological functional connectivity patterns, combining static sparse design with dynamic pruning and rewiring mechanisms.

Findings

G2GNet achieves up to 75% sparsity with improved accuracy.

Outperforms dense models on standard vision benchmarks.

Reduces parameters and computation significantly.

Abstract

The structure of biological neural circuits-modular, hierarchical, and sparsely interconnected-reflects an efficient trade-off between wiring cost, functional specialization, and robustness. These principles offer valuable insights for artificial neural network (ANN) design, especially as networks grow in depth and scale. Sparsity, in particular, has been widely explored for reducing memory and computation, improving speed, and enhancing generalization. Motivated by systems neuroscience findings, we explore how patterns of functional connectivity in the mouse visual cortex-specifically, ensemble-to-ensemble communication, can inform ANN design. We introduce G2GNet, a novel architecture that imposes sparse, modular connectivity across feedforward layers. Despite having significantly fewer parameters than fully connected models, G2GNet achieves superior accuracy on standard vision benchmarks. To our knowledge, this is the first architecture to incorporate biologically observed functional connectivity patterns as a structural bias in ANN design. We complement this static bias with a dynamic sparse training (DST) mechanism that prunes and regrows edges during training. We also propose a Hebbian-inspired rewiring rule based on activation correlations, drawing on principles of biological plasticity. G2GNet achieves up to 75% sparsity while improving accuracy by up to 4.3% on benchmarks, including Fashion-MNIST, CIFAR-10, and CIFAR-100, outperforming dense baselines with far fewer computations.