Learning beyond sensations: how dreams organize neuronal representations

TL;DR

This paper explores how dreaming and virtual experiences contribute to organizing neuronal representations in the brain, proposing new learning principles that extend beyond traditional sensory-based models.

Contribution

It introduces two novel principles, adversarial dreaming and contrastive dreaming, to explain how virtual experiences shape cortical representations.

Findings

Dreams may support adversarial learning in cortex.

Virtual experiences help achieve invariance in neuronal representations.

The proposed principles align with known cortical structures and sleep phenomenology.

Abstract

Semantic representations in higher sensory cortices form the basis for robust, yet flexible behavior. These representations are acquired over the course of development in an unsupervised fashion and continuously maintained over an organism's lifespan. Predictive learning theories propose that these representations emerge from predicting or reconstructing sensory inputs. However, brains are known to generate virtual experiences, such as during imagination and dreaming, that go beyond previously experienced inputs. Here, we suggest that virtual experiences may be just as relevant as actual sensory inputs in shaping cortical representations. In particular, we discuss two complementary learning principles that organize representations through the generation of virtual experiences. First, "adversarial dreaming" proposes that creative dreams support a cortical implementation of adversarial learning in which feedback and feedforward pathways engage in a productive game of trying to fool each other. Second, "contrastive dreaming" proposes that the invariance of neuronal representations to irrelevant factors of variation is acquired by trying to map similar virtual experiences together via a contrastive learning process. These principles are compatible with known cortical structure and dynamics and the phenomenology of sleep thus providing promising directions to explain cortical learning beyond the classical predictive learning paradigm.