Meta-learning three-factor plasticity rules for structured credit assignment with sparse feedback

TL;DR

This paper introduces a meta-learning framework that discovers biologically plausible local plasticity rules enabling recurrent neural networks to learn from sparse, delayed feedback through structured credit assignment.

Contribution

It presents a novel meta-learning approach that optimizes local three-factor learning rules for recurrent networks, bridging the gap between biological plausibility and effective reinforcement learning.

Findings

Learned rules support credit assignment with sparse, delayed rewards.

Rules are biologically plausible, relying on local information.

Framework generalizes to different tasks and network architectures.

Abstract

Biological neural networks learn complex behaviors from sparse, delayed feedback using local synaptic plasticity, yet the mechanisms enabling structured credit assignment remain elusive. In contrast, artificial recurrent networks solving similar tasks typically rely on biologically implausible global learning rules or hand-crafted local updates. The space of local plasticity rules capable of supporting learning from delayed reinforcement remains largely unexplored. Here, we present a meta-learning framework that discovers local learning rules for structured credit assignment in recurrent networks trained with sparse feedback. Our approach interleaves local neo-Hebbian-like updates during task execution with an outer loop that optimizes plasticity parameters via \textbf{tangent-propagation through learning}. The resulting three-factor learning rules enable long-timescale credit assignment using only local information and delayed rewards, offering new insights into biologically grounded mechanisms for learning in recurrent circuits.