Deviant Learning Algorithm: Learning Sparse Mismatch Representations through Time and Space

TL;DR

This paper introduces the Deviant Learning Algorithm, a bio-mimetic approach inspired by predictive coding and mismatch negativity, which enhances neural learning systems' performance with sparse representations and large synaptic networks.

Contribution

The paper proposes a novel Deviant Learning Algorithm that integrates predictive coding and mismatch negativity concepts to improve neural network learning and decision-making.

Findings

Achieves competitive predictions with small datasets

Utilizes large numbers of synapses for better performance

Incorporates biological insights into computational models

Abstract

Predictive coding (PDC) has recently attracted attention in the neuroscience and computing community as a candidate unifying paradigm for neuronal studies and artificial neural network implementations particularly targeted at unsupervised learning systems. The Mismatch Negativity (MMN) has also recently been studied in relation to PC and found to be a useful ingredient in neural predictive coding systems. Backed by the behavior of living organisms, such networks are particularly useful in forming spatio-temporal transitions and invariant representations of the input world. However, most neural systems still do not account for large number of synapses even though this has been shown by a few machine learning researchers as an effective and very important component of any neural system if such a system is to behave properly. Our major point here is that PDC systems with the MMN effect in addition to a large number of synapses can greatly improve any neural learning system's performance and ability to make decisions in the machine world. In this paper, we propose a novel bio-mimetic computational intelligence algorithm -- the Deviant Learning Algorithm, inspired by these key ideas and functional properties of recent brain-cognitive discoveries and theories. We also show by numerical experiments guided by theoretical insights, how our invented bio-mimetic algorithm can achieve competitive predictions even with very small problem specific data.