Synaptic balancing: a biologically plausible local learning rule that provably increases neural network noise robustness without sacrificing task performance

TL;DR

This paper presents a biologically plausible local learning rule that enhances neural network noise robustness without compromising task performance, linking neurobiology, network engineering, and mathematical systems.

Contribution

It introduces a novel synaptic balancing learning rule that preserves network function and improves noise robustness, grounded in integrable dynamical systems theory.

Findings

Increases noise robustness without performance loss

Maintains entire input-output trajectory mappings

Aligns with experimental heterosynaptic plasticity observations

Abstract

We introduce a novel, biologically plausible local learning rule that provably increases the robustness of neural dynamics to noise in nonlinear recurrent neural networks with homogeneous nonlinearities. Our learning rule achieves higher noise robustness without sacrificing performance on the task and without requiring any knowledge of the particular task. The plasticity dynamics -- an integrable dynamical system operating on the weights of the network -- maintains a multiplicity of conserved quantities, most notably the network's entire temporal map of input to output trajectories. The outcome of our learning rule is a synaptic balancing between the incoming and outgoing synapses of every neuron. This synaptic balancing rule is consistent with many known aspects of experimentally observed heterosynaptic plasticity, and moreover makes new experimentally testable predictions relating plasticity at the incoming and outgoing synapses of individual neurons. Overall, this work provides a novel, practical local learning rule that exactly preserves overall network function and, in doing so, provides new conceptual bridges between the disparate worlds of the neurobiology of heterosynaptic plasticity, the engineering of regularized noise-robust networks, and the mathematics of integrable Lax dynamical systems.