Parallel Learning by Multitasking Neural Networks

TL;DR

This paper demonstrates how a modified Hebbian neural network can learn multiple patterns simultaneously, revealing different regimes of parallel and hierarchical processing, with results supported by theoretical and simulation methods.

Contribution

It introduces the Multitasking Hebbian Network capable of parallel pattern learning, extending associative neural network models to multitasking scenarios with theoretical and empirical validation.

Findings

Network handles multiple patterns in parallel with different regimes.

Performance unaffected by supervised or unsupervised training in low-storage.

Results align across statistical mechanics, signal-to-noise, and Monte Carlo simulations.

Abstract

A modern challenge of Artificial Intelligence is learning multiple patterns at once (i.e.parallel learning). While this can not be accomplished by standard Hebbian associative neural networks, in this paper we show how the Multitasking Hebbian Network (a variation on theme of the Hopfield model working on sparse data-sets) is naturally able to perform this complex task. We focus on systems processing in parallel a finite (up to logarithmic growth in the size of the network) amount of patterns, mirroring the low-storage level of standard associative neural networks at work with pattern recognition. For mild dilution in the patterns, the network handles them hierarchically, distributing the amplitudes of their signals as power-laws w.r.t. their information content (hierarchical regime), while, for strong dilution, all the signals pertaining to all the patterns are raised with the same strength (parallel regime). Further, confined to the low-storage setting (i.e., far from the spin glass limit), the presence of a teacher neither alters the multitasking performances nor changes the thresholds for learning: the latter are the same whatever the training protocol is supervised or unsupervised. Results obtained through statistical mechanics, signal-to-noise technique and Monte Carlo simulations are overall in perfect agreement and carry interesting insights on multiple learning at once: for instance, whenever the cost-function of the model is minimized in parallel on several patterns (in its description via Statistical Mechanics), the same happens to the standard sum-squared error Loss function (typically used in Machine Learning).