Diffusion of Neuromodulators for Temporal Credit Assignment

TL;DR

This paper proposes a biologically inspired diffusion-based learning mechanism for neural networks, enabling effective temporal credit assignment in sparse feedback scenarios, and demonstrates its advantages on benchmark tasks.

Contribution

It introduces a novel diffusion-based neuromodulatory learning method that enhances credit assignment in recurrent spiking neural networks with sparse feedback.

Findings

Diffusive credit signaling improves learning performance.

The method outperforms baseline eligibility propagation.

Applicable to biologically plausible neural models.

Abstract

Biological learning achieves temporal credit assignment despite sparse and imprecise feedback, often relying on neuromodulatory signals acting over space and time. Here, we introduce a learning mechanism in which error information diffuses locally through the network, similar to volume transmission of neuromodulators. This distributed modulation allows neurons to learn even in the absence of direct feedback, using the local concentration of the diffusing credit signal. Applied to recurrent spiking neural networks with sparse feedback connectivity, diffusive credit signaling improves learning across three benchmark tasks. Using eligibility propagation as a baseline learning mechanism, we show how diffusion-based modulation can provide a plausible mechanism for credit assignment in sparsely connected neural circuits.