Towards biologically plausible Dreaming and Planning in recurrent spiking networks

TL;DR

This paper introduces a biologically plausible spiking neural network model that uses dreaming and planning to improve learning efficiency without extensive experience storage, aligning with biological processes.

Contribution

It presents a novel two-module spiking neural network that employs dreaming and planning for online learning, enhancing biological plausibility and hardware implementability.

Findings

Dreaming significantly boosts learning performance.

Planning achieves comparable results to dreaming.

Model learns online without storing detailed experiences.

Abstract

Humans and animals can learn new skills after practicing for a few hours, while current reinforcement learning algorithms require a large amount of data to achieve good performances. Recent model-based approaches show promising results by reducing the number of necessary interactions with the environment to learn a desirable policy. However, these methods require biological implausible ingredients, such as the detailed storage of older experiences, and long periods of offline learning. The optimal way to learn and exploit word-models is still an open question. Taking inspiration from biology, we suggest that dreaming might be an efficient expedient to use an inner model. We propose a two-module (agent and model) spiking neural network in which "dreaming" (living new experiences in a model-based simulated environment) significantly boosts learning. We also explore "planning", an online alternative to dreaming, that shows comparable performances. Importantly, our model does not require the detailed storage of experiences, and learns online the world-model and the policy. Moreover, we stress that our network is composed of spiking neurons, further increasing the biological plausibility and implementability in neuromorphic hardware.