Dreaming Learning
Alessandro Londei, Matteo Benati, Denise Lanzieri, Vittorio Loreto

TL;DR
Dreaming Learning introduces a novel training algorithm inspired by the Adjacent Possible concept, enabling neural networks to better adapt to non-stationary data and regime shifts by exploring new data spaces during training.
Contribution
The paper proposes a new training method that anticipates regime shifts and adapts to non-stationary data, improving model responsiveness and convergence speed.
Findings
Improved auto-correlation of textual sequences by approximately 29%.
Enhanced loss convergence velocity by around 100% during paradigm shifts.
Demonstrated effectiveness on Markov chains and non-stationary textual data.
Abstract
Incorporating novelties into deep learning systems remains a challenging problem. Introducing new information to a machine learning system can interfere with previously stored data and potentially alter the global model paradigm, especially when dealing with non-stationary sources. In such cases, traditional approaches based on validation error minimization offer limited advantages. To address this, we propose a training algorithm inspired by Stuart Kauffman's notion of the Adjacent Possible. This novel training methodology explores new data spaces during the learning phase. It predisposes the neural network to smoothly accept and integrate data sequences with different statistical characteristics than expected. The maximum distance compatible with such inclusion depends on a specific parameter: the sampling temperature used in the explorative phase of the present method. This…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSleep and Wakefulness Research · Child Therapy and Development
