Input anticipating critical reservoirs show power law forgetting of unexpected input events
Norbert Michael Mayer

TL;DR
This paper explores how echo state networks can exhibit power law forgetting of input events when operating at critical connectivity and using specific input anticipation, extending the typical exponential decay in reservoir computing.
Contribution
It demonstrates conditions under which reservoir networks can achieve power law memory decay, focusing on critical connectivity and input anticipation strategies.
Findings
Power law forgetting is achievable with critical connectivity.
Restricted recurrent connectivity supports long-term memory.
Input anticipation influences memory decay behavior.
Abstract
Usually, reservoir computing shows an exponential memory decay. This paper investigates under which circumstances echo state networks can show a power law forgetting. That means traces of earlier events can be found in the reservoir for very long time spans. Such a setting requires critical connectivity exactly at the limit of what is permissible according the echo state condition. However, for general matrices the limit cannot be determined exactly from theory. In addition, the behavior of the network is strongly influenced by the input flow. Results are presented that use certain types of restricted recurrent connectivity and anticipation learning with regard to the input, where indeed power law forgetting can be achieved.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Reservoir Computing · Neural dynamics and brain function · Neural Networks and Applications
