Cognitive computation with autonomously active neural networks: an emerging field

TL;DR

This paper reviews theoretical models of autonomously active neural networks, focusing on architectures like saddle point networks and attractor relics, which perform ongoing cognitive processing such as independent component analysis.

Contribution

It introduces a novel perspective on neural network architectures that balance stability and sensitivity, enabling autonomous, ongoing cognitive functions.

Findings

Networks of attractor relics can balance stability and responsiveness.

Unsupervised Hebbian learning links internal states with sensory input.

The system performs continuous independent component analysis autonomously.

Abstract

The human brain is autonomously active. To understand the functional role of this self-sustained neural activity, and its interplay with the sensory data input stream, is an important question in cognitive system research and we review here the present state of theoretical modelling. This review will start with a brief overview of the experimental efforts, together with a discussion of transient vs. self-sustained neural activity in the framework of reservoir computing. The main emphasis will be then on two paradigmal neural network architectures showing continuously ongoing transient-state dynamics: saddle point networks and networks of attractor relics. Self-active neural networks are confronted with two seemingly contrasting demands: a stable internal dynamical state and sensitivity to incoming stimuli. We show, that this dilemma can be solved by networks of attractor relics based on competitive neural dynamics, where the attractor relics compete on one side with each other for transient dominance, and on the other side with the dynamical influence of the input signals. Unsupervised and local Hebbian-style online learning then allows the system to build up correlations between the internal dynamical transient states and the sensory input stream. An emergent cognitive capability results from this set-up. The system performs online, and on its own, a non-linear independent component analysis of the sensory data stream, all the time being continuously and autonomously active. This process maps the independent components of the sensory input onto the attractor relics, which acquire in this way a semantic meaning.