Dynamical Equations With Bottom-up Self-Organizing Properties Learn Accurate Dynamical Hierarchies Without Any Loss Function

TL;DR

This paper introduces a novel dynamical system based on nonlinear feedback loops that self-organizes to learn hierarchical structures from sequential data without using any loss function, surpassing existing unsupervised methods.

Contribution

It presents a new approach where dynamical equations with feedback loops enable self-organization and hierarchical learning without traditional loss functions, offering insights into cognition and pattern recognition.

Findings

Outperforms state-of-the-art unsupervised algorithms in most experiments.

Learns hierarchical structures from sequential data effectively.

Exhibits adaptive phase transition-like phenomena in response to input changes.

Abstract

Self-organization is ubiquitous in nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that general patterns are difficult to define in terms of dynamical equations and designing a system that could learn by reordering itself is still to be seen. Here, we propose a learning system, where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops, allowing attractor-repeller pairs to emerge for each pattern observed. Experiments reveal that such a system can map temporal to spatial correlation, enabling hierarchical structures to be learned from sequential data. The results are accurate enough to surpass state-of-the-art unsupervised learning algorithms in seven out of eight experiments as well as two real-world problems. Interestingly, the dynamic nature of the system makes it inherently adaptive, giving rise to phenomena similar to phase transitions in chemistry/thermodynamics when the input structure changes. Thus, the work here sheds light on how self-organization can allow for pattern recognition and hints at how intelligent behavior might emerge from simple dynamic equations without any objective/loss function.