Robots that learn to evaluate models of collective behavior

TL;DR

This paper presents a reinforcement-learning framework using a biomimetic robotic fish to evaluate and compare computational models of live fish behavior through closed-loop interaction, revealing the most accurate models.

Contribution

It introduces a novel embodied evaluation method that quantitatively assesses behavioral model fidelity via real-time robotic interaction with live animals.

Findings

Neural network-based fish model showed the smallest sim-to-real gap.

The framework effectively distinguishes model accuracy under closed-loop conditions.

The approach uncovers deficiencies in existing behavioral models.

Abstract

Understanding and modeling animal behavior is essential for studying collective motion, decision-making, and bio-inspired robotics. Yet, evaluating the accuracy of behavioral models still often relies on offline comparisons to static trajectory statistics. Here we introduce a reinforcement-learning-based framework that uses a biomimetic robotic fish (RoboFish) to evaluate computational models of live fish behavior through closed-loop interaction. We trained policies in simulation using four distinct fish models-a simple constant-follow baseline, two rule-based models, and a biologically grounded convolutional neural network model-and transferred these policies to the real RoboFish setup, where they interacted with live fish. Policies were trained to guide a simulated fish to goal locations, enabling us to quantify how the response of real fish differs from the simulated fish's response. We evaluate the fish models by quantifying the sim-to-real gaps, defined as the Wasserstein distance between simulated and real distributions of behavioral metrics such as goal-reaching performance, inter-individual distances, wall interactions, and alignment. The neural network-based fish model exhibited the smallest gap across goal-reaching performance and most other metrics, indicating higher behavioral fidelity than conventional rule-based models under this benchmark. More importantly, this separation shows that the proposed evaluation can quantitatively distinguish candidate models under matched closed-loop conditions. Our work demonstrates how learning-based robotic experiments can uncover deficiencies in behavioral models and provides a general framework for evaluating animal behavior models through embodied interaction.