Hebbian Learning with Global Direction

TL;DR

This paper introduces a novel Global-guided Hebbian Learning framework that combines local Hebbian updates with global signals, improving scalability and performance on large-scale tasks like ImageNet.

Contribution

The paper proposes a biologically inspired, model-agnostic framework that integrates local Hebbian learning with global guidance, enhancing scalability and effectiveness.

Findings

Outperforms existing Hebbian methods across various tasks.

Achieves competitive results on ImageNet, narrowing the gap with backpropagation.

Demonstrates the effectiveness of global signals in guiding Hebbian updates.

Abstract

Backpropagation algorithm has driven the remarkable success of deep neural networks, but its lack of biological plausibility and high computational costs have motivated the ongoing search for alternative training methods. Hebbian learning has attracted considerable interest as a biologically plausible alternative to backpropagation. Nevertheless, its exclusive reliance on local information, without consideration of global task objectives, fundamentally limits its scalability. Inspired by the biological synergy between neuromodulators and local plasticity, we introduce a novel model-agnostic Global-guided Hebbian Learning (GHL) framework, which seamlessly integrates local and global information to scale up across diverse networks and tasks. In specific, the local component employs Oja's rule with competitive learning to ensure stable and effective local updates. Meanwhile, the global component introduces a sign-based signal that guides the direction of local Hebbian plasticity updates. Extensive experiments demonstrate that our method consistently outperforms existing Hebbian approaches. Notably, on large-scale network and complex datasets like ImageNet, our framework achieves the competitive results and significantly narrows the gap with standard backpropagation.