Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines

TL;DR

This paper introduces Ornstein-Uhlenbeck adaptation (OUA), a novel noise-based learning mechanism that operates continuously and locally, offering an alternative to gradient descent for biological and artificial systems, with broad applicability.

Contribution

The paper proposes OUA, a new learning method using adaptive noise processes, applicable to diverse tasks and capable of meta-learning, bridging biological plausibility and machine learning.

Findings

OUA performs well in supervised and reinforcement learning tasks.

It can autonomously adjust hyper-parameters through meta-learning.

OUA offers a biologically plausible alternative to gradient-based learning.

Abstract

Learning is a fundamental property of intelligent systems, observed across biological organisms and engineered systems. While modern intelligent systems typically rely on gradient descent for learning, the need for exact gradients and complex information flow makes its implementation in biological and neuromorphic systems challenging. This has motivated the exploration of alternative learning mechanisms that can operate locally and do not rely on exact gradients. In this work, we introduce a novel approach that leverages noise in the parameters of the system and global reinforcement signals. Using an Ornstein-Uhlenbeck process with adaptive dynamics, our method balances exploration and exploitation during learning, driven by deviations from error predictions, akin to reward prediction error. Operating in continuous time, Orstein-Uhlenbeck adaptation (OUA) is proposed as a general mechanism for learning dynamic, time-evolving environments. We validate our approach across diverse tasks, including supervised learning and reinforcement learning in feedforward and recurrent systems. Additionally, we demonstrate that it can perform meta-learning, adjusting hyper-parameters autonomously. Our results indicate that OUA provides a viable alternative to traditional gradient-based methods, with potential applications in neuromorphic computing. It also hints at a possible mechanism for noise-driven learning in the brain, where stochastic neurotransmitter release may guide synaptic adjustments.