# From Crystallized Adaptivity to Fluid Adaptivity in Deep Reinforcement Learning -- Insights from Biological Systems on Adaptive Flexibility

## TL;DR

This paper explores how biological systems achieve fluid adaptivity in changing environments and discusses strategies to incorporate similar flexibility into deep reinforcement learning systems.

## Contribution

It introduces the concept of fluid adaptivity, contrasting it with traditional crystallized adaptivity, and proposes research strategies inspired by biological systems to enhance AI flexibility.

## Key findings

- Biological systems demonstrate fluid adaptivity in dynamic environments.
- Dynamizing the problem space can improve AI adaptability.
- Neuronal modeling offers insights into flexible learning mechanisms.

## Abstract

Recent developments in machine-learning algorithms have led to impressive performance increases in many traditional application scenarios of artificial intelligence research. In the area of deep reinforcement learning, deep learning functional architectures are combined with incremental learning schemes for sequential tasks that include interaction-based, but often delayed feedback. Despite their impressive successes, modern machine-learning approaches, including deep reinforcement learning, still perform weakly when compared to flexibly adaptive biological systems in certain naturally occurring scenarios. Such scenarios include transfers to environments different than the ones in which the training took place or environments that dynamically change, both of which are often mastered by biological systems through a capability that we here term "fluid adaptivity" to contrast it from the much slower adaptivity ("crystallized adaptivity") of the prior learning from which the behavior emerged. In this article, we derive and discuss research strategies, based on analyzes of fluid adaptivity in biological systems and its neuronal modeling, that might aid in equipping future artificially intelligent systems with capabilities of fluid adaptivity more similar to those seen in some biologically intelligent systems. A key component of this research strategy is the dynamization of the problem space itself and the implementation of this dynamization by suitably designed flexibly interacting modules.