A Deep Reinforcement Learning Framework for Closed-loop Guidance of Fish Schools via Virtual Agents

TL;DR

This paper introduces a deep reinforcement learning framework using virtual agents to guide fish schools in real-time, demonstrating effectiveness in small groups but facing challenges with larger ones.

Contribution

It presents a novel deep RL approach for closed-loop guidance of live fish using simulation-trained policies and evaluates its performance in physical experiments.

Findings

White background and larger stimuli improve guidance efficacy.

Effective guidance achieved for groups of five fish.

Guidance performance declines with groups of eight fish.

Abstract

Guiding collective motion in biological groups is a fundamental challenge in understanding social interaction rules and developing automated systems for animal management. In this study, we propose a deep reinforcement learning (RL) framework for the closed-loop guidance of fish schools using virtual agents. These agents are controlled by policies trained via Proximal Policy Optimization (PPO) in simulation and deployed in physical experiments with rummy-nose tetras (Petitella bleheri), enabling real-time interaction between artificial agents and live individuals. To cope with the stochastic behavior of live individuals, we design a composite reward function to balance directional guidance with social cohesion. Our systematic evaluation of visual parameters shows that a white background and larger stimulus sizes maximize guidance efficacy in physical trials. Furthermore, evaluation across group sizes revealed that while the system demonstrates effective guidance for groups of five individuals, this capability markedly degrades as group size increases to eight. This study highlights the potential of deep RL for automated guidance of biological collectives and identifies challenges in maintaining artificial influence in larger groups.