What Neuroscience Can Teach AI About Learning in Continuously Changing Environments

TL;DR

This paper explores how insights from neuroscience about animals' ability to adapt to changing environments can inform and improve AI systems' continual learning capabilities, especially in dynamic real-world settings.

Contribution

It integrates neuroscience and AI literature to propose a research agenda for enhancing AI's adaptability based on biological learning mechanisms.

Findings

Neuroscience reveals rapid behavioral and neuronal shifts in response to environmental changes.

Current AI models lack the quick adaptation seen in biological systems.

A proposed research agenda bridges neuroscience insights with AI development for continual learning.

Abstract

Modern AI models, such as large language models, are usually trained once on a huge corpus of data, potentially fine-tuned for a specific task, and then deployed with fixed parameters. Their training is costly, slow, and gradual, requiring billions of repetitions. In stark contrast, animals continuously adapt to the ever-changing contingencies in their environments. This is particularly important for social species, where behavioral policies and reward outcomes may frequently change in interaction with peers. The underlying computational processes are often marked by rapid shifts in an animal's behaviour and rather sudden transitions in neuronal population activity. Such computational capacities are of growing importance for AI systems operating in the real world, like those guiding robots or autonomous vehicles, or for agentic AI interacting with humans online. Can AI learn from neuroscience? This Perspective explores this question, integrating the literature on continual and in-context learning in AI with the neuroscience of learning on behavioral tasks with shifting rules, reward probabilities, or outcomes. We will outline an agenda for how specifically insights from neuroscience may inform current developments in AI in this area, and - vice versa - what neuroscience may learn from AI, contributing to the evolving field of NeuroAI.