Abrupt and spontaneous strategy switches emerge in simple regularised neural networks
Anika T. L\"owe, L\'eo Touzo, Paul S. Muhle-Karbe, Andrew M. Saxe,, Christopher Summerfield, Nicolas W. Schuck

TL;DR
This study demonstrates that simple regularised neural networks can exhibit sudden insight-like strategy switches similar to human 'aha-moments', driven by noise, gating, and regularisation during gradual learning.
Contribution
It shows that insight-like behaviour can emerge naturally in simple neural networks through specific mechanisms, challenging the view that insights require complex cognition.
Findings
Humans show sudden strategy switches linked to insights.
Neural networks mimic human insight-like switches under certain conditions.
Gating, noise, and regularisation are crucial for emergent insight behaviour.
Abstract
Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, whose behaviour was marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural dynamics and brain function · Aesthetic Perception and Analysis · Visual perception and processing mechanisms
