On-line learning in a discrete state space

TL;DR

This paper investigates online learning in a discrete state space for an N-dimensional Ising perceptron, revealing limitations for small L and success for larger L, specifically around rom the abstract, it explores the impact of state space size on learning efficacy.

Contribution

It demonstrates how the size of the discrete state space influences the ability of Hebbian learning to achieve finite overlap in online learning scenarios.

Findings

No finite overlap for L=2 in the thermodynamic limit.

Finite overlap achieved when L is on the order of or larger L, specifically or rom the abstract, it explores the impact of state space size on learning efficacy.

It demonstrates how the size of the discrete state space influences the ability of Hebbian learning to achieve finite overlap in online learning scenarios.

Abstract

On-line learning of a rule given by an N-dimensional Ising perceptron, is considered for the case when the student is constrained to take values in a discrete state space of size $L^N$. For L=2 no on-line algorithm can achieve a finite overlap with the teacher in the thermodynamic limit. However, if $L$ is on the order of $\sqrt{N}$, Hebbian learning does achieve a finite overlap.