Visual novelty, curiosity, and intrinsic reward in machine learning and the brain

TL;DR

This paper reviews how visual novelty functions as an intrinsic reward in machine learning and the brain, highlighting parallels between computational algorithms and neural mechanisms that promote exploration and learning.

Contribution

It proposes that in the visual system, novelty representations are used for flexible generalization to enhance learning, bridging machine learning and neuroscience insights.

Findings

Novelty-driven algorithms improve exploration efficiency.

Visual novelty signals are used for flexible generalization.

Parallels between machine learning and primate brain mechanisms.

Abstract

A strong preference for novelty emerges in infancy and is prevalent across the animal kingdom. When incorporated into reinforcement-based machine learning algorithms, visual novelty can act as an intrinsic reward signal that vastly increases the efficiency of exploration and expedites learning, particularly in situations where external rewards are difficult to obtain. Here we review parallels between recent developments in novelty-driven machine learning algorithms and our understanding of how visual novelty is computed and signaled in the primate brain. We propose that in the visual system, novelty representations are not configured with the principal goal of detecting novel objects, but rather with the broader goal of flexibly generalizing novelty information across different states in the service of driving novelty-based learning.