FishGym: A High-Performance Physics-based Simulation Framework for Underwater Robot Learning

TL;DR

FishGym is a high-performance simulation platform designed for training underwater fish-like robots using GPU-accelerated fluid-structure interaction, enabling efficient data generation for reinforcement learning in complex aquatic environments.

Contribution

The paper introduces FishGym, a novel GPU-based simulation framework for underwater robots that accurately models fluid-structure interactions for reinforcement learning.

Findings

Successfully trained control policies for multiple benchmark tasks.

Generated realistic fish swimming behaviors and trajectories.

Demonstrated advantages over empirical models.

Abstract

Bionic underwater robots have demonstrated their superiority in many applications. Yet, training their intelligence for a variety of tasks that mimic the behavior of underwater creatures poses a number of challenges in practice, mainly due to lack of a large amount of available training data as well as the high cost in real physical environment. Alternatively, simulation has been considered as a viable and important tool for acquiring datasets in different environments, but it mostly targeted rigid and soft body systems. There is currently dearth of work for more complex fluid systems interacting with immersed solids that can be efficiently and accurately simulated for robot training purposes. In this paper, we propose a new platform called "FishGym", which can be used to train fish-like underwater robots. The framework consists of a robotic fish modeling module using articulated body with skinning, a GPU-based high-performance localized two-way coupled fluid-structure interaction simulation module that handles both finite and infinitely large domains, as well as a reinforcement learning module. We leveraged existing training methods with adaptations to underwater fish-like robots and obtained learned control policies for multiple benchmark tasks. The training results are demonstrated with reasonable motion trajectories, with comparisons and analyses to empirical models as well as known real fish swimming behaviors to highlight the advantages of the proposed platform.