BEAN: Interpretable Representation Learning with Biologically-Enhanced Artificial Neuronal Assembly Regularization

TL;DR

This paper introduces BEAN, a regularization method inspired by neuroscience that improves interpretability, efficiency, and generalization of deep neural networks by modeling neuronal dependencies akin to biological neuronal assemblies.

Contribution

The paper proposes a novel BEAN regularization that models neuronal correlations, enhancing interpretability, efficiency, and few-shot learning capabilities of neural networks.

Findings

BEAN enables formation of interpretable neuronal clusters.

BEAN promotes sparse and efficient networks without performance loss.

BEAN improves generalization in few-shot learning scenarios.

Abstract

Deep neural networks (DNNs) are known for extracting useful information from large amounts of data. However, the representations learned in DNNs are typically hard to interpret, especially in dense layers. One crucial issue of the classical DNN model such as multilayer perceptron (MLP) is that neurons in the same layer of DNNs are conditionally independent of each other, which makes co-training and emergence of higher modularity difficult. In contrast to DNNs, biological neurons in mammalian brains display substantial dependency patterns. Specifically, biological neural networks encode representations by so-called neuronal assemblies: groups of neurons interconnected by strong synaptic interactions and sharing joint semantic content. The resulting population coding is essential for human cognitive and mnemonic processes. Here, we propose a novel Biologically Enhanced Artificial Neuronal assembly (BEAN) regularization to model neuronal correlations and dependencies, inspired by cell assembly theory from neuroscience. Experimental results show that BEAN enables the formation of interpretable neuronal functional clusters and consequently promotes a sparse, memory/computation-efficient network without loss of model performance. Moreover, our few-shot learning experiments demonstrate that BEAN could also enhance the generalizability of the model when training samples are extremely limited.