Neuromodulation via Krotov-Hopfield Improves Accuracy and Robustness of RBMs

TL;DR

This paper introduces a neuromodulation technique based on Krotov-Hopfield dynamics that enhances the accuracy, robustness, and generalization of Restricted Boltzmann Machines (RBMs) in reconstruction and classification tasks.

Contribution

It demonstrates that Krotov-Hopfield-modulated RBMs outperform standard RBMs in accuracy, robustness, and efficiency, introducing a biologically inspired neuromodulation approach to improve neural network performance.

Findings

KH-modulated RBMs outperform standard RBMs in reconstruction tasks.

KH-modulated RBMs show improved classification accuracy.

KH-modulated RBMs are more robust to weight initialization and overfitting.

Abstract

In biological systems, neuromodulation tunes synaptic plasticity based on the internal state of the organism, complementing stimulus-driven Hebbian learning. The algorithm recently proposed by Krotov and Hopfield \cite{krotov_2019} can be utilized to mirror this process in artificial neural networks, where its built-in intra-layer competition and selective inhibition of synaptic updates offer a cost-effective remedy for the lack of lateral connections through a simplified attention mechanism. We demonstrate that KH-modulated RBMs outperform standard (shallow) RBMs in both reconstruction and classification tasks, offering a superior trade-off between generalization performance and model size, with the additional benefit of robustness to weight initialization as well as to overfitting during training.