Input Statistics and Hebbian Crosstalk Effects

TL;DR

This paper examines how inspecific Hebbian learning, influenced by input statistics, affects neural learning outcomes, revealing that input nature critically determines the impact of synaptic cross-talk and questioning the existence of universal correction mechanisms.

Contribution

It extends the Oja model to analyze inspecific Hebbian learning across various input patterns, highlighting the dependence of cross-talk effects on input statistics and the potential need for proofreading mechanisms.

Findings

Inspecificity effects vary strongly with input statistics.

Cross-talk can cause dramatic learning inaccuracies.

Universal correction algorithms are unlikely to exist.

Abstract

As an extension of prior work, we study inspecific Hebbian learning using the classical Oja model. We use a combination of analytical tools and numerical simulations to investigate how the effects of inspecificity (or synaptic "cross-talk") depend on the input statistics. We investigated a variety of patterns that appear in dimensions higher than 2 (and classified them based on covariance type and input bias). The effects of inspecificity on the learning outcome were found to depend very strongly on the nature of the input, and in some cases were very dramatic, making unlikely the existence of a generic neural algorithm to correct learning inaccuracy due to cross-talk. We discuss the possibility that sophisticated learning, such as presumably occurs in the neocortex, is enabled as much by special proofreading machinery for enhancing specificity, as by special algorithms.