Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron

TL;DR

This paper develops a stochastic-process framework to analyze how nonlinearity and data distribution influence learning dynamics in nonlinear perceptrons under supervised and reinforcement learning, with implications for biological and artificial neural networks.

Contribution

It introduces a novel flow equation approach to study nonlinear perceptron learning, accounting for data noise and task interference, extending beyond linear and simplified models.

Findings

Data noise impacts learning speed differently in SL and RL.

Input-data distribution affects how quickly tasks are forgotten.

The approach is validated with MNIST dataset.

Abstract

The ability of a brain or a neural network to efficiently learn depends crucially on both the task structure and the learning rule. Previous works have analyzed the dynamical equations describing learning in the relatively simplified context of the perceptron under assumptions of a student-teacher framework or a linearized output. While these assumptions have facilitated theoretical understanding, they have precluded a detailed understanding of the roles of the nonlinearity and input-data distribution in determining the learning dynamics, limiting the applicability of the theories to real biological or artificial neural networks. Here, we use a stochastic-process approach to derive flow equations describing learning, applying this framework to the case of a nonlinear perceptron performing binary classification. We characterize the effects of the learning rule (supervised or reinforcement learning, SL/RL) and input-data distribution on the perceptron's learning curve and the forgetting curve as subsequent tasks are learned. In particular, we find that the input-data noise differently affects the learning speed under SL vs. RL, as well as determines how quickly learning of a task is overwritten by subsequent learning. Additionally, we verify our approach with real data using the MNIST dataset. This approach points a way toward analyzing learning dynamics for more-complex circuit architectures.