Optimising low-Reynolds-number predation via optimal control and reinforcement learning

TL;DR

This paper combines optimal control and reinforcement learning to identify and reproduce the most efficient predation strategies for a microswimmer predator at low Reynolds number, optimizing for time and energy efficiency.

Contribution

It introduces a novel approach using optimal control and RL to determine and replicate optimal predation strategies for microswimmers, considering prey size and flow effects.

Findings

Two-fold L-shaped trajectory accelerates predation.

Stresslet mode improves predation speed and efficiency.

RL successfully reproduces optimal predation paths.

Abstract

We seek the best stroke sequences of a finite-size swimming predator chasing a non-motile point or finite--size prey at low Reynolds number. We use optimal control to seek the globally-optimal solutions for the former and RL for general situations. The predator is represented by a squirmer model that can translate forward and laterally, rotate and generate a stresslet flow. We identify the predator's best squirming sequences to achieve the time-optimal (TO) and efficiency-optimal (EO) predation. For a point prey, the TO squirmer executing translational motions favours a two-fold L-shaped trajectory that enables it to exploit the disturbance flow for accelerated predation; using a stresslet mode significantly expedites the EO predation, allowing the predator to catch the prey faster yet with lower energy consumption and higher predatory efficiency; the predator can harness its stresslet disturbance flow to suck the prey towards itself; compared to a translating predator, its compeer combining translation and rotation is less time--efficient, and the latter occasionally achieves the TO predation via retreating in order to advance. We also adopt RL to reproduce the globally-optimal predatory strategy of chasing a point prey, qualitatively capturing the crucial two--fold attribute of TO path. Using a numerically emulated RL environment, we explore the dependence of the optimal predatory path on the size of prey. Our results might provide useful information that help design synthetic microswimmers such as \textit{in vivo} medical micro-robots capable of capturing and approaching objects in viscous flows.