Obstacle-aware navigation of smart microswimmers in a turbulent flow

TL;DR

This paper develops an obstacle-aware reinforcement learning approach for microswimmers navigating turbulent flows with obstacles, improving their ability to reach targets efficiently.

Contribution

It extends existing Q-learning methods to account for obstacles, enabling microswimmers to avoid trapping and improve navigation in turbulent environments.

Findings

Obstacle-aware microswimmers outperform naive and surfing strategies.

The method suppresses trapping near obstacles in turbulent flows.

Enhanced navigation efficiency demonstrated in 2D Navier-Stokes turbulence simulations.

Abstract

Microswimmers in turbulent flows often navigate complex, heterogeneous, and obstacle-rich environments, where they exhibit intricate behaviors such as trapping at and escape from obstacles. We generalize recent $\mathcal{Q}-$learning methods of J.K. Alageshan \textit{et al.} [Phys.Rev.E \textbf{101}, 043110 (2020)] and A. Gupta \textit{et al.} [Physics of Fluids \textbf{37}, 045107 (2025)] developed for non-interacting microswimmers that aim to move optimally from an initial position to a target, to account for the additional complication of an obstacle in the flow. We begin by considering one circular obstacle in forced two-dimensional (2D) Navier-Stokes turbulence in which the energy spectrum displays a forward cascade. We employ the volume-penalization method to introduce this obstacle within our doubly periodic simulation domain. We augment our adversarial $\mathcal{Q}-$learning Refs.~\cite{Alageshan_2020,Akanksha_2025} by suppressing the tendency of microswimmers to get trapped in stagnation points in the vicinity of the obstacle. We demonstrate that smart microswimmers ($SS$), which adopt our obstacle-aware adversarial $\mathcal{Q}-$learning strategy, outperform both na\"ive swimmers ($NS$) and surfers ($SuS$).