Semantic learning in autonomously active recurrent neural networks

TL;DR

This paper explores how autonomous recurrent neural networks can learn semantic correlations between internal activity and external stimuli through transient state dynamics and unsupervised diffusive learning signals.

Contribution

It introduces a novel framework where autonomous neural activity learns semantic relations via transient states and a diffusive learning signal during sensitive transition periods.

Findings

The model performs nonlinear independent component analysis autonomously.

External stimuli influence internal dynamics during transition periods.

The system demonstrates emergent cognitive capabilities from neural ensemble competition.

Abstract

The human brain is autonomously active, being characterized by a self-sustained neural activity which would be present even in the absence of external sensory stimuli. Here we study the interrelation between the self-sustained activity in autonomously active recurrent neural nets and external sensory stimuli. There is no a priori semantical relation between the influx of external stimuli and the patterns generated internally by the autonomous and ongoing brain dynamics. The question then arises when and how are semantic correlations between internal and external dynamical processes learned and built up? We study this problem within the paradigm of transient state dynamics for the neural activity in recurrent neural nets, i.e. for an autonomous neural activity characterized by an infinite time-series of transiently stable attractor states. We propose that external stimuli will be relevant during the sensitive periods, {\it viz} the transition period between one transient state and the subsequent semi-stable attractor. A diffusive learning signal is generated unsupervised whenever the stimulus influences the internal dynamics qualitatively. For testing we have presented to the model system stimuli corresponding to the bars and stripes problem. We found that the system performs a non-linear independent component analysis on its own, being continuously and autonomously active. This emergent cognitive capability results here from a general principle for the neural dynamics, the competition between neural ensembles.