Deep reinforcement learning for tracking a moving target in jellyfish-like swimming

TL;DR

This paper presents a deep reinforcement learning approach to control a flexible, jellyfish-like swimmer for effective target tracking in fluid flows, incorporating action regulation to handle fluid-structure interactions.

Contribution

It introduces a novel deep Q-network framework with action regulation for controlling a flexible swimmer in fluid environments, extending machine learning applications in bio-inspired fluid dynamics.

Findings

The swimmer successfully tracks moving targets using the DQN controller.

Action regulation improves navigation accuracy amidst complex fluid interactions.

The method demonstrates potential for bio-inspired robotic control in fluid environments.

Abstract

We develop a deep reinforcement learning method for training a jellyfish-like swimmer to effectively track a moving target in a two-dimensional flow. This swimmer is a flexible object equipped with a muscle model based on torsional springs. We employ a deep Q-network (DQN) that takes the swimmer's geometry and dynamic parameters as inputs, and outputs actions which are the forces applied to the swimmer. In particular, we introduce an action regulation to mitigate the interference from complex fluid-structure interactions. The goal of these actions is to navigate the swimmer to a target point in the shortest possible time. In the DQN training, the data on the swimmer's motions are obtained from simulations conducted using the immersed boundary method. During tracking a moving target, there is an inherent delay between the application of forces and the corresponding response of the swimmer's body due to hydrodynamic interactions between the shedding vortices and the swimmer's own locomotion. Our tests demonstrate that the swimmer, with the DQN agent and action regulation, is able to dynamically adjust its course based on its instantaneous state. This work extends the application scope of machine learning in controlling flexible objects within fluid environments.