Deep reinforcement learning based navigation of a jellyfish-like swimmer in flows with obstacles

TL;DR

This paper introduces a deep reinforcement learning approach for controlling a bio-inspired jellyfish swimmer to navigate complex obstacle-filled flows, emphasizing the importance of force feedback for improved obstacle avoidance.

Contribution

The study develops a physics-aware RL framework that incorporates real-time force and torque feedback, enhancing navigation efficiency in fluid environments with obstacles.

Findings

Force feedback improves obstacle avoidance performance.

The augmented state leads to smoother, earlier maneuvers.

Enhanced wall interaction exploitation for navigation.

Abstract

We develop a deep reinforcement learning framework for controlling a bio-inspired jellyfish swimmer to navigate complex fluid environments with obstacles. While existing methods often rely on kinematic and geometric states, a key challenge remains in achieving efficient obstacle avoidance under strong fluid-structure interactions and near-wall effects. We augment the agent's state representation within a soft actor-critic algorithm to include the real-time forces and torque experienced by the swimmer, providing direct mechanical feedback from vortex-wall interactions. This augmented state space enables the swimmer to perceive and interpret wall proximity and orientation through distinct hydrodynamic force signatures. We analyze how these force and torque patterns, generated by walls at different positions influence the swimmer's decision-making policy. Comparative experiments with a baseline model without force feedback demonstrate that the present one with force feedback achieves higher navigation efficiency in two-dimensional obstacle-avoidance tasks. The results show that explicit force feedback facilitates earlier, smoother maneuvers and enables the exploitation of wall effects for efficient turning behaviors. With an application to autonomous cave mapping, this work underscores the critical role of direct mechanical feedback in fluid environments and presents a physics-aware machine learning framework for advancing robust underwater exploration systems.