Self-shaping dynamical systems and learning

TL;DR

This paper introduces self-shaping dynamical systems that adapt their vector fields in response to stimuli, enabling learning and pattern recognition without traditional neural network constraints.

Contribution

It proposes a mathematical framework for self-shaping systems that evolve their vector fields and phase space, generalizing neural networks and offering new insights into learning processes.

Findings

Self-shaping systems develop deterministic vector fields from random stimuli.

They can perform pattern recognition without supervision.

They reconstruct input probability distributions without rigid units.

Abstract

We associate learning and adaptation in living systems with the shaping of the velocity vector field in the respective dynamical systems in response to external, generally random, stimuli. With this, a mathematical concept of self-shaping dynamical systems is proposed. Initially there is a zero vector field and an "empty" phase space with no attractors or other non-trivial objects. As the random stimulus begins, the vector field deforms and eventually becomes smooth and deterministic, despite the random nature of the applied force, while the phase space develops various geometrical objects. We consider gradient self-shaping systems, whose vector field is the gradient of some energy function, which under certain conditions develops into the multi-dimensional probability density distribution (PDD) of the input. Self-shaping systems are relevant to neural networks (NNs) of two types: Hopfield, and probabilistic. Firstly, we show that they can potentially perform pattern recognition tasks traditionally delegated to Hopfield NNs, but without supervision and on-line, and without developing spurious minima of the energy. Secondly, like probabilistic NNs, they can reconstruct the PDD of input signals, without the limitation that new training patterns have to enter as new hardware units. Thus, self-shaping systems can be regarded as a generalization of the NN concept, achieved by abandoning the "rigid units" - "flexible couplings" paradigm and making the vector field fully flexible and amenable to external force. The new concept presents an engineering challenge requiring new principles of hardware design. It might also become an alternative paradigm for modeling of living and learning systems.