Hocus-Socus: An Error Catastrophe for Complex Hebbian Learning Implies Neocortical Proofreading

TL;DR

This paper investigates the limitations of Hebbian learning in neurons for detecting complex higher-order correlations and proposes a proofreading mechanism in the neocortex to enable such learning, crucial for intelligence.

Contribution

It reveals that nonlinear Hebbian learning fails for complex correlations without connection-specificity and introduces a cortical proofreading hypothesis to overcome this.

Findings

Hebbian learning defaults to simpler correlations without specificity

Neurons struggle to learn higher-order correlations in models

Proposed cortical proofreading mechanism to enable complex learning

Abstract

The neocortex is widely believed to be the seat of intelligence and "mind". However, it's unclear what "mind" is, or how the special features of neocortex enable it, though likely "connectionist" principles are involved *A. The key to intelligence1 is learning relationships between large numbers of signals (such as pixel values), rather than memorizing explicit patterns. Causes (such as objects) can then be inferred from a learned internal model. These relationships fall into 2 classes: simple pairwise or second-order correlations (socs), and complex, and vastly more numerous, higher-order correlations (hocsB), such as the product of 3 or more pixels averaged over a set of images. Thus if 3 pixels correlate, they may give an "edge". Neurons with "Hebbian" synapses (changing strength in response to input-output spike-coincidences) are sensitive to such correlations, and it's likely that learned internal models use such neurons. Because output firing depends on input firing via the relevant connection strengths, Hebbian learning provides, in a feedback manner, sensitivity to input correlations. Hocs are vital, since they express "interesting" structure2 (e.g. edges), but their detection requires nonlinear rules operating at synapses of individual neurons. Here we report that in single model neurons learning from hocs fails, and defaults to socs, if nonlinear Hebbian rules are not sufficiently connection-specific. Such failure would inevitably occur if a neuron's input synapses were too crowded, and would undermine biological connectionism. Since the cortex must be hoc-sensitive to achieve the type of learning enabling mind, we propose it uses known, detailed but poorly understood circuitry and physiology to "proofread" Hebbian connections. Analogous DNA proofreading allows evolution of complex genomes (i.e. "life").